[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781864224577058-825-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 10:17:04 dist-test-slave-6pt2 krb5kdc[832](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 10:17:04 dist-test-slave-6pt2 krb5kdc[832](info): set up 2 sockets
Jun 19 10:17:04 dist-test-slave-6pt2 krb5kdc[832](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260619 10:17:05.602936 825 mini_kdc.cc:121] Time spent starting KDC: real 1.008s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781864224577058-825-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781864224577058-825-0/krb5kdc/test-user.keytab.
Jun 19 10:17:05 dist-test-slave-6pt2 krb5kdc[832](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864225, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T10:17:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T10:17:05Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.206.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.206.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.206.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781864224577058-825-0/krb5kdc/kudu_127.0.206.126.keytab.
Entry for principal kudu/127.0.206.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781864224577058-825-0/krb5kdc/kudu_127.0.206.126.keytab.
W20260619 10:17:06.007746 825 mini_kdc.cc:262] Time spent creating service keytab for kudu/127.0.206.126: real 0.315s user 0.002s sys 0.001s
I20260619 10:17:06.008203 825 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTU9OP6/build/release/bin/kudu
/tmp/dist-test-taskTU9OP6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781864224577058-825-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781864224577058-825-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781864224577058-825-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781864224577058-825-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.206.126:43441
--webserver_interface=127.0.206.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781864224577058-825-0/krb5kdc/kudu_127.0.206.126.keytab
--principal=kudu/127.0.206.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.206.84:33171
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.206.126:43441
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781864224577058-825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781864224577058-825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781864224577058-825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 10:17:06.087651 845 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 10:17:06.087865 845 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 10:17:06.087888 845 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 10:17:06.089061 845 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 10:17:06.089103 845 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 10:17:06.089118 845 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 10:17:06.089130 845 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 10:17:06.089143 845 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 10:17:06.090675 845 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.206.84:33171
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781864224577058-825-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781864224577058-825-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.206.126:43441
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781864224577058-825-0/krb5kdc/kudu_127.0.206.126.keytab
--principal=kudu/127.0.206.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.206.126:43441
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781864224577058-825-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.206.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.845
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781864224577058-825-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 09:32:44 UTC on ccycloud.quasar-gbtldp.root.comops.site
build id 4455440
I20260619 10:17:06.090935 845 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 10:17:06.091183 845 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 10:17:06.096047 853 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 10:17:06.096073 851 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 10:17:06.096081 850 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 10:17:06.096407 845 server_base.cc:1034] running on GCE node
I20260619 10:17:06.096611 845 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 10:17:06.096889 845 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 10:17:06.098027 845 hybrid_clock.cc:648] HybridClock initialized: now 1781864226098015 us; error 30 us; skew 500 ppm
Jun 19 10:17:06 dist-test-slave-6pt2 krb5kdc[832](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864226, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 10:17:06.100385 845 init.cc:377] Logged in from keytab as kudu/127.0.206.126@KRBTEST.COM (short username kudu)
I20260619 10:17:06.100889 845 webserver.cc:460] Webserver started at http://127.0.206.126:36155/ using document root <none> and password file <none>
I20260619 10:17:06.101130 845 fs_manager.cc:359] Metadata directory not provided
I20260619 10:17:06.101194 845 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 10:17:06.101321 845 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 10:17:06.102447 845 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781864224577058-825-0/minicluster-data/master-0/data/instance:
uuid: "a457beee792546f1800d7c8fc8219614"
format_stamp: "Formatted at 2026-06-19 10:17:06 on dist-test-slave-6pt2"
server_key: "5b9f95a38f953f931a77f8ec36c7bec1"
server_key_iv: "09ba41a27872a6358c4373df39628ea7"
server_key_version: "encryptionkey@0"
I20260619 10:17:06.102866 845 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781864224577058-825-0/minicluster-data/master-0/wal/instance:
uuid: "a457beee792546f1800d7c8fc8219614"
format_stamp: "Formatted at 2026-06-19 10:17:06 on dist-test-slave-6pt2"
server_key: "5b9f95a38f953f931a77f8ec36c7bec1"
server_key_iv: "09ba41a27872a6358c4373df39628ea7"
server_key_version: "encryptionkey@0"
I20260619 10:17:06.104646 845 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.002s
I20260619 10:17:06.105420 860 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 10:17:06.105610 845 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260619 10:17:06.105707 845 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781864224577058-825-0/minicluster-data/master-0/data,/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781864224577058-825-0/minicluster-data/master-0/wal
uuid: "a457beee792546f1800d7c8fc8219614"
format_stamp: "Formatted at 2026-06-19 10:17:06 on dist-test-slave-6pt2"
server_key: "5b9f95a38f953f931a77f8ec36c7bec1"
server_key_iv: "09ba41a27872a6358c4373df39628ea7"
server_key_version: "encryptionkey@0"
I20260619 10:17:06.105814 845 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781864224577058-825-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781864224577058-825-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781864224577058-825-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 10:17:06.127444 845 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 10:17:06.129349 845 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 10:17:06.129534 845 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 10:17:06.133687 845 rpc_server.cc:307] RPC server started. Bound to: 127.0.206.126:43441
I20260619 10:17:06.133741 912 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.206.126:43441 every 8 connection(s)
I20260619 10:17:06.134114 845 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781864224577058-825-0/minicluster-data/master-0/data/info.pb
I20260619 10:17:06.134984 913 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 10:17:06.137192 913 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a457beee792546f1800d7c8fc8219614: Bootstrap starting.
I20260619 10:17:06.137763 913 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a457beee792546f1800d7c8fc8219614: Neither blocks nor log segments found. Creating new log.
I20260619 10:17:06.138043 913 log.cc:824] T 00000000000000000000000000000000 P a457beee792546f1800d7c8fc8219614: Log is configured to *not* fsync() on all Append() calls
I20260619 10:17:06.138775 913 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a457beee792546f1800d7c8fc8219614: No bootstrap required, opened a new log
I20260619 10:17:06.140560 913 raft_consensus.cc:348] T 00000000000000000000000000000000 P a457beee792546f1800d7c8fc8219614 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a457beee792546f1800d7c8fc8219614" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 43441 } }
I20260619 10:17:06.140710 913 raft_consensus.cc:374] T 00000000000000000000000000000000 P a457beee792546f1800d7c8fc8219614 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 10:17:06.140734 913 raft_consensus.cc:729] T 00000000000000000000000000000000 P a457beee792546f1800d7c8fc8219614 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a457beee792546f1800d7c8fc8219614, State: Initialized, Role: FOLLOWER
I20260619 10:17:06.140900 913 consensus_queue.cc:260] T 00000000000000000000000000000000 P a457beee792546f1800d7c8fc8219614 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a457beee792546f1800d7c8fc8219614" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 43441 } }
I20260619 10:17:06.140959 913 raft_consensus.cc:388] T 00000000000000000000000000000000 P a457beee792546f1800d7c8fc8219614 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 10:17:06.141017 913 raft_consensus.cc:482] T 00000000000000000000000000000000 P a457beee792546f1800d7c8fc8219614 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 10:17:06.141065 913 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a457beee792546f1800d7c8fc8219614 [term 0 FOLLOWER]: Advancing to term 1
I20260619 10:17:06.141606 913 raft_consensus.cc:504] T 00000000000000000000000000000000 P a457beee792546f1800d7c8fc8219614 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a457beee792546f1800d7c8fc8219614" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 43441 } }
I20260619 10:17:06.141721 913 leader_election.cc:302] T 00000000000000000000000000000000 P a457beee792546f1800d7c8fc8219614 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a457beee792546f1800d7c8fc8219614; no voters:
I20260619 10:17:06.141896 913 leader_election.cc:288] T 00000000000000000000000000000000 P a457beee792546f1800d7c8fc8219614 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 10:17:06.141932 916 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a457beee792546f1800d7c8fc8219614 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 10:17:06.142091 916 raft_consensus.cc:686] T 00000000000000000000000000000000 P a457beee792546f1800d7c8fc8219614 [term 1 LEADER]: Becoming Leader. State: Replica: a457beee792546f1800d7c8fc8219614, State: Running, Role: LEADER
I20260619 10:17:06.142161 913 sys_catalog.cc:564] T 00000000000000000000000000000000 P a457beee792546f1800d7c8fc8219614 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 10:17:06.142174 916 consensus_queue.cc:237] T 00000000000000000000000000000000 P a457beee792546f1800d7c8fc8219614 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a457beee792546f1800d7c8fc8219614" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 43441 } }
I20260619 10:17:06.142724 917 sys_catalog.cc:455] T 00000000000000000000000000000000 P a457beee792546f1800d7c8fc8219614 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a457beee792546f1800d7c8fc8219614" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a457beee792546f1800d7c8fc8219614" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 43441 } } }
I20260619 10:17:06.142810 917 sys_catalog.cc:458] T 00000000000000000000000000000000 P a457beee792546f1800d7c8fc8219614 [sys.catalog]: This master's current role is: LEADER
I20260619 10:17:06.142969 825 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTU9OP6/build/release/bin/kudu as pid 845
I20260619 10:17:06.143064 825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781864224577058-825-0/minicluster-data/master-0/wal/instance
I20260619 10:17:06.143224 925 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 10:17:06.143745 925 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 10:17:06.143898 825 external_mini_cluster.cc:1428] Setting key 71b5bf89a5bf15b9305dd2c61ced94eb
I20260619 10:17:06.143874 918 sys_catalog.cc:455] T 00000000000000000000000000000000 P a457beee792546f1800d7c8fc8219614 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a457beee792546f1800d7c8fc8219614. Latest consensus state: current_term: 1 leader_uuid: "a457beee792546f1800d7c8fc8219614" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a457beee792546f1800d7c8fc8219614" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 43441 } } }
I20260619 10:17:06.144178 918 sys_catalog.cc:458] T 00000000000000000000000000000000 P a457beee792546f1800d7c8fc8219614 [sys.catalog]: This master's current role is: LEADER
I20260619 10:17:06.147073 925 catalog_manager.cc:1347] Generated new cluster ID: 773c2850953d4891bfd12f02459ed471
I20260619 10:17:06.147125 925 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Jun 19 10:17:06 dist-test-slave-6pt2 krb5kdc[832](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864225, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.206.126@KRBTEST.COM
I20260619 10:17:06.161069 934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:06.144779 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36260 (local address 127.0.206.126:43441)
0619 10:17:06.144966 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0619 10:17:06.144975 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 10:17:06.145258 (+ 283us) server_negotiation.cc:407] Connection header received
0619 10:17:06.146910 (+ 1652us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:17:06.146916 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:17:06.146976 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:17:06.147105 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:17:06.149106 (+ 2001us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:06.150715 (+ 1609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:06.152149 (+ 1434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:06.152735 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:06.152853 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:06.157068 (+ 4215us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 10:17:06.157072 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 10:17:06.157072 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 10:17:06.157084 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0619 10:17:06.159057 (+ 1973us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:17:06.159566 (+ 509us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:17:06.159569 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:17:06.159570 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:17:06.159682 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:17:06.159981 (+ 299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:17:06.159982 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:17:06.159983 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:17:06.160276 (+ 293us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 10:17:06.160411 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0619 10:17:06.160723 (+ 312us) server_negotiation.cc:299] Negotiation successful
0619 10:17:06.160854 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":148,"thread_start_us":86,"threads_started":1}
I20260619 10:17:06.161561 925 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 10:17:06.162223 925 catalog_manager.cc:1504] Loading token signing keys...
I20260619 10:17:06.167644 925 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a457beee792546f1800d7c8fc8219614: Generated new TSK 0
I20260619 10:17:06.167836 925 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.0.206.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.206.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.206.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781864224577058-825-0/krb5kdc/kudu_127.0.206.65.keytab.
Entry for principal kudu/127.0.206.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781864224577058-825-0/krb5kdc/kudu_127.0.206.65.keytab.
I20260619 10:17:06.222992 825 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTU9OP6/build/release/bin/kudu
/tmp/dist-test-taskTU9OP6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781864224577058-825-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781864224577058-825-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781864224577058-825-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781864224577058-825-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.206.65:0
--local_ip_for_outbound_sockets=127.0.206.65
--webserver_interface=127.0.206.65
--webserver_port=0
--tserver_master_addrs=127.0.206.126:43441
--keytab_file=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781864224577058-825-0/krb5kdc/kudu_127.0.206.65.keytab
--principal=kudu/127.0.206.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.206.84:33171
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781864224577058-825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781864224577058-825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781864224577058-825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 10:17:06.302439 939 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 10:17:06.302680 939 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 10:17:06.302707 939 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 10:17:06.303821 939 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 10:17:06.303859 939 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 10:17:06.303911 939 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.206.65
I20260619 10:17:06.305310 939 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.206.84:33171
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781864224577058-825-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781864224577058-825-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781864224577058-825-0/krb5kdc/kudu_127.0.206.65.keytab
--principal=kudu/127.0.206.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.206.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781864224577058-825-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.206.65
--webserver_port=0
--tserver_master_addrs=127.0.206.126:43441
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.939
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.206.65
--log_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781864224577058-825-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 09:32:44 UTC on ccycloud.quasar-gbtldp.root.comops.site
build id 4455440
I20260619 10:17:06.305536 939 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 10:17:06.305747 939 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 10:17:06.310567 945 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 10:17:06.310557 947 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 10:17:06.310631 944 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 10:17:06.311424 939 server_base.cc:1034] running on GCE node
I20260619 10:17:06.311587 939 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 10:17:06.311772 939 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 10:17:06.312918 939 hybrid_clock.cc:648] HybridClock initialized: now 1781864226312895 us; error 38 us; skew 500 ppm
Jun 19 10:17:06 dist-test-slave-6pt2 krb5kdc[832](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864226, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 10:17:06.315382 939 init.cc:377] Logged in from keytab as kudu/127.0.206.65@KRBTEST.COM (short username kudu)
I20260619 10:17:06.315878 939 webserver.cc:460] Webserver started at http://127.0.206.65:39995/ using document root <none> and password file <none>
I20260619 10:17:06.316088 939 fs_manager.cc:359] Metadata directory not provided
I20260619 10:17:06.316154 939 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 10:17:06.316303 939 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 10:17:06.317260 939 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781864224577058-825-0/minicluster-data/ts-0/data/instance:
uuid: "c0b9030fc6c346d58e8fd2bbeb39a8f8"
format_stamp: "Formatted at 2026-06-19 10:17:06 on dist-test-slave-6pt2"
server_key: "e2240a5b81ad72d4a8cb30315487b9bd"
server_key_iv: "fe4e8467df786860b6f2d0e8f691af04"
server_key_version: "encryptionkey@0"
I20260619 10:17:06.317610 939 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781864224577058-825-0/minicluster-data/ts-0/wal/instance:
uuid: "c0b9030fc6c346d58e8fd2bbeb39a8f8"
format_stamp: "Formatted at 2026-06-19 10:17:06 on dist-test-slave-6pt2"
server_key: "e2240a5b81ad72d4a8cb30315487b9bd"
server_key_iv: "fe4e8467df786860b6f2d0e8f691af04"
server_key_version: "encryptionkey@0"
I20260619 10:17:06.318902 939 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260619 10:17:06.319561 954 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 10:17:06.319737 939 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260619 10:17:06.319811 939 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781864224577058-825-0/minicluster-data/ts-0/data,/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781864224577058-825-0/minicluster-data/ts-0/wal
uuid: "c0b9030fc6c346d58e8fd2bbeb39a8f8"
format_stamp: "Formatted at 2026-06-19 10:17:06 on dist-test-slave-6pt2"
server_key: "e2240a5b81ad72d4a8cb30315487b9bd"
server_key_iv: "fe4e8467df786860b6f2d0e8f691af04"
server_key_version: "encryptionkey@0"
I20260619 10:17:06.319891 939 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781864224577058-825-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781864224577058-825-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781864224577058-825-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 10:17:06.328965 939 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 10:17:06.330713 939 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 10:17:06.330857 939 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 10:17:06.331066 939 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 10:17:06.331359 939 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 10:17:06.331391 939 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 10:17:06.331444 939 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 10:17:06.331470 939 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 10:17:06.337924 939 rpc_server.cc:307] RPC server started. Bound to: 127.0.206.65:46657
I20260619 10:17:06.337976 1067 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.206.65:46657 every 8 connection(s)
I20260619 10:17:06.338272 939 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781864224577058-825-0/minicluster-data/ts-0/data/info.pb
Jun 19 10:17:06 dist-test-slave-6pt2 krb5kdc[832](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864226, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.65@KRBTEST.COM for kudu/127.0.206.126@KRBTEST.COM
I20260619 10:17:06.346781 934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:06.339000 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.65:37881 (local address 127.0.206.126:43441)
0619 10:17:06.339045 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0619 10:17:06.339047 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 10:17:06.339218 (+ 171us) server_negotiation.cc:407] Connection header received
0619 10:17:06.339686 (+ 468us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:17:06.339687 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:17:06.339696 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:17:06.339752 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:17:06.340643 (+ 891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:06.341151 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:06.342144 (+ 993us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:06.342334 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:06.342385 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:06.344523 (+ 2138us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 10:17:06.344525 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 10:17:06.344525 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 10:17:06.344532 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 10:17:06.345663 (+ 1131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:17:06.346011 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:17:06.346012 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:17:06.346012 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:17:06.346057 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:17:06.346253 (+ 196us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:17:06.346254 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:17:06.346254 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:17:06.346345 (+ 91us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 10:17:06.346392 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0619 10:17:06.346642 (+ 250us) server_negotiation.cc:299] Negotiation successful
0619 10:17:06.346719 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260619 10:17:06.346983 1070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:06.338958 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:43441 (local address 127.0.206.65:37881)
0619 10:17:06.339110 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0619 10:17:06.339149 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0619 10:17:06.339604 (+ 455us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 10:17:06.339810 (+ 206us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 10:17:06.339814 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 10:17:06.340396 (+ 582us) client_negotiation.cc:189] Negotiated authn=SASL
0619 10:17:06.340568 (+ 172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:06.340572 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:06.341225 (+ 653us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:06.341225 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:06.342071 (+ 846us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:06.342073 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:06.342402 (+ 329us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:06.342403 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:06.342495 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:06.343024 (+ 529us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 10:17:06.343034 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0619 10:17:06.344442 (+ 1408us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 10:17:06.345731 (+ 1289us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 10:17:06.345736 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 10:17:06.345738 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 10:17:06.345942 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 10:17:06.346113 (+ 171us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 10:17:06.346113 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 10:17:06.346113 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 10:17:06.346202 (+ 89us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 10:17:06.346398 (+ 196us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 10:17:06.346402 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 10:17:06.346502 (+ 100us) client_negotiation.cc:769] Sending connection context
0619 10:17:06.346578 (+ 76us) client_negotiation.cc:240] Negotiation successful
0619 10:17:06.346746 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":87,"threads_started":1}
I20260619 10:17:06.347335 825 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTU9OP6/build/release/bin/kudu as pid 939
I20260619 10:17:06.347426 825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781864224577058-825-0/minicluster-data/ts-0/wal/instance
I20260619 10:17:06.347498 1068 heartbeater.cc:347] Connected to a master server at 127.0.206.126:43441
I20260619 10:17:06.347587 1068 heartbeater.cc:464] Registering TS with master...
I20260619 10:17:06.347589 825 external_mini_cluster.cc:1428] Setting key c80e2071ab8758fe82e11a1b7ead9397
I20260619 10:17:06.347769 1068 heartbeater.cc:511] Master 127.0.206.126:43441 requested a full tablet report, sending...
I20260619 10:17:06.348376 877 ts_manager.cc:194] Registered new tserver with Master: c0b9030fc6c346d58e8fd2bbeb39a8f8 (127.0.206.65:46657)
I20260619 10:17:06.348938 877 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.206.65@KRBTEST.COM'} at 127.0.206.65:37881
WARNING: no policy specified for kudu/127.0.206.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.206.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.206.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781864224577058-825-0/krb5kdc/kudu_127.0.206.66.keytab.
Entry for principal kudu/127.0.206.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781864224577058-825-0/krb5kdc/kudu_127.0.206.66.keytab.
I20260619 10:17:06.368857 825 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTU9OP6/build/release/bin/kudu
/tmp/dist-test-taskTU9OP6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781864224577058-825-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781864224577058-825-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781864224577058-825-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781864224577058-825-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.206.66:0
--local_ip_for_outbound_sockets=127.0.206.66
--webserver_interface=127.0.206.66
--webserver_port=0
--tserver_master_addrs=127.0.206.126:43441
--keytab_file=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781864224577058-825-0/krb5kdc/kudu_127.0.206.66.keytab
--principal=kudu/127.0.206.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.206.84:33171
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781864224577058-825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781864224577058-825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781864224577058-825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 10:17:06.446952 1073 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 10:17:06.447177 1073 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 10:17:06.447216 1073 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 10:17:06.448340 1073 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 10:17:06.448406 1073 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 10:17:06.448449 1073 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.206.66
I20260619 10:17:06.449810 1073 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.206.84:33171
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781864224577058-825-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781864224577058-825-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781864224577058-825-0/krb5kdc/kudu_127.0.206.66.keytab
--principal=kudu/127.0.206.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.206.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781864224577058-825-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.206.66
--webserver_port=0
--tserver_master_addrs=127.0.206.126:43441
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1073
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.206.66
--log_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781864224577058-825-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 09:32:44 UTC on ccycloud.quasar-gbtldp.root.comops.site
build id 4455440
I20260619 10:17:06.450066 1073 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 10:17:06.450333 1073 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 10:17:06.455008 1079 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 10:17:06.455010 1081 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 10:17:06.455097 1078 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 10:17:06.455222 1073 server_base.cc:1034] running on GCE node
I20260619 10:17:06.455449 1073 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 10:17:06.455704 1073 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 10:17:06.456872 1073 hybrid_clock.cc:648] HybridClock initialized: now 1781864226456872 us; error 38 us; skew 500 ppm
Jun 19 10:17:06 dist-test-slave-6pt2 krb5kdc[832](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864226, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 10:17:06.459203 1073 init.cc:377] Logged in from keytab as kudu/127.0.206.66@KRBTEST.COM (short username kudu)
I20260619 10:17:06.459649 1073 webserver.cc:460] Webserver started at http://127.0.206.66:40135/ using document root <none> and password file <none>
I20260619 10:17:06.459867 1073 fs_manager.cc:359] Metadata directory not provided
I20260619 10:17:06.459930 1073 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 10:17:06.460063 1073 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 10:17:06.461073 1073 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781864224577058-825-0/minicluster-data/ts-1/data/instance:
uuid: "de6ffe28d50b419b854e95dafceb093c"
format_stamp: "Formatted at 2026-06-19 10:17:06 on dist-test-slave-6pt2"
server_key: "572460b37a5822bdb32435513c501d3e"
server_key_iv: "d3351dbf415cde6d1fef02d5bd64a56d"
server_key_version: "encryptionkey@0"
I20260619 10:17:06.461439 1073 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781864224577058-825-0/minicluster-data/ts-1/wal/instance:
uuid: "de6ffe28d50b419b854e95dafceb093c"
format_stamp: "Formatted at 2026-06-19 10:17:06 on dist-test-slave-6pt2"
server_key: "572460b37a5822bdb32435513c501d3e"
server_key_iv: "d3351dbf415cde6d1fef02d5bd64a56d"
server_key_version: "encryptionkey@0"
I20260619 10:17:06.462745 1073 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260619 10:17:06.463442 1088 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 10:17:06.463610 1073 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260619 10:17:06.463701 1073 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781864224577058-825-0/minicluster-data/ts-1/data,/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781864224577058-825-0/minicluster-data/ts-1/wal
uuid: "de6ffe28d50b419b854e95dafceb093c"
format_stamp: "Formatted at 2026-06-19 10:17:06 on dist-test-slave-6pt2"
server_key: "572460b37a5822bdb32435513c501d3e"
server_key_iv: "d3351dbf415cde6d1fef02d5bd64a56d"
server_key_version: "encryptionkey@0"
I20260619 10:17:06.463799 1073 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781864224577058-825-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781864224577058-825-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781864224577058-825-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 10:17:06.479684 1073 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 10:17:06.481297 1073 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 10:17:06.481455 1073 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 10:17:06.481652 1073 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 10:17:06.481984 1073 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 10:17:06.482051 1073 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 10:17:06.482084 1073 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 10:17:06.482110 1073 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 10:17:06.488683 1073 rpc_server.cc:307] RPC server started. Bound to: 127.0.206.66:45697
I20260619 10:17:06.488716 1201 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.206.66:45697 every 8 connection(s)
I20260619 10:17:06.489008 1073 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781864224577058-825-0/minicluster-data/ts-1/data/info.pb
I20260619 10:17:06.492857 825 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTU9OP6/build/release/bin/kudu as pid 1073
I20260619 10:17:06.492962 825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781864224577058-825-0/minicluster-data/ts-1/wal/instance
I20260619 10:17:06.493122 825 external_mini_cluster.cc:1428] Setting key 7d0e4a9950720897990e1f7b167a3714
Jun 19 10:17:06 dist-test-slave-6pt2 krb5kdc[832](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864226, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.66@KRBTEST.COM for kudu/127.0.206.126@KRBTEST.COM
I20260619 10:17:06.498768 934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:06.489651 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.66:56799 (local address 127.0.206.126:43441)
0619 10:17:06.489708 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0619 10:17:06.489710 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 10:17:06.489898 (+ 188us) server_negotiation.cc:407] Connection header received
0619 10:17:06.490401 (+ 503us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:17:06.490403 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:17:06.490415 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:17:06.490464 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:17:06.491420 (+ 956us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:06.491900 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:06.493034 (+ 1134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:06.493328 (+ 294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:06.493366 (+ 38us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:06.495923 (+ 2557us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 10:17:06.495924 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 10:17:06.495925 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 10:17:06.495931 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 10:17:06.497315 (+ 1384us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:17:06.497773 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:17:06.497774 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:17:06.497774 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:17:06.497822 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:17:06.498108 (+ 286us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:17:06.498108 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:17:06.498108 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:17:06.498199 (+ 91us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 10:17:06.498246 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0619 10:17:06.498570 (+ 324us) server_negotiation.cc:299] Negotiation successful
0619 10:17:06.498674 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260619 10:17:06.499001 1204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:06.489622 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:43441 (local address 127.0.206.66:56799)
0619 10:17:06.489781 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0619 10:17:06.489823 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0619 10:17:06.490292 (+ 469us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 10:17:06.490526 (+ 234us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 10:17:06.490530 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 10:17:06.491134 (+ 604us) client_negotiation.cc:189] Negotiated authn=SASL
0619 10:17:06.491323 (+ 189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:06.491327 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:06.491990 (+ 663us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:06.491991 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:06.492910 (+ 919us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:06.492913 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:06.493462 (+ 549us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:06.493463 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:06.493586 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:06.494136 (+ 550us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 10:17:06.494144 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 10:17:06.495826 (+ 1682us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 10:17:06.497401 (+ 1575us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 10:17:06.497407 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 10:17:06.497409 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 10:17:06.497700 (+ 291us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 10:17:06.497892 (+ 192us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 10:17:06.497893 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 10:17:06.497893 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 10:17:06.498027 (+ 134us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 10:17:06.498267 (+ 240us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 10:17:06.498273 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 10:17:06.498433 (+ 160us) client_negotiation.cc:769] Sending connection context
0619 10:17:06.498551 (+ 118us) client_negotiation.cc:240] Negotiation successful
0619 10:17:06.498743 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":78,"threads_started":1}
I20260619 10:17:06.499452 1202 heartbeater.cc:347] Connected to a master server at 127.0.206.126:43441
I20260619 10:17:06.499544 1202 heartbeater.cc:464] Registering TS with master...
I20260619 10:17:06.499714 1202 heartbeater.cc:511] Master 127.0.206.126:43441 requested a full tablet report, sending...
I20260619 10:17:06.500057 877 ts_manager.cc:194] Registered new tserver with Master: de6ffe28d50b419b854e95dafceb093c (127.0.206.66:45697)
I20260619 10:17:06.500422 877 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.206.66@KRBTEST.COM'} at 127.0.206.66:56799
WARNING: no policy specified for kudu/127.0.206.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.206.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.206.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781864224577058-825-0/krb5kdc/kudu_127.0.206.67.keytab.
Entry for principal kudu/127.0.206.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781864224577058-825-0/krb5kdc/kudu_127.0.206.67.keytab.
I20260619 10:17:06.516551 825 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTU9OP6/build/release/bin/kudu
/tmp/dist-test-taskTU9OP6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781864224577058-825-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781864224577058-825-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781864224577058-825-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781864224577058-825-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.206.67:0
--local_ip_for_outbound_sockets=127.0.206.67
--webserver_interface=127.0.206.67
--webserver_port=0
--tserver_master_addrs=127.0.206.126:43441
--keytab_file=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781864224577058-825-0/krb5kdc/kudu_127.0.206.67.keytab
--principal=kudu/127.0.206.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.206.84:33171
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781864224577058-825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781864224577058-825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781864224577058-825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 10:17:06.595302 1207 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 10:17:06.595476 1207 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 10:17:06.595497 1207 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 10:17:06.596627 1207 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 10:17:06.596669 1207 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 10:17:06.596724 1207 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.206.67
I20260619 10:17:06.598130 1207 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.206.84:33171
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781864224577058-825-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781864224577058-825-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781864224577058-825-0/krb5kdc/kudu_127.0.206.67.keytab
--principal=kudu/127.0.206.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.206.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781864224577058-825-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.206.67
--webserver_port=0
--tserver_master_addrs=127.0.206.126:43441
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1207
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.206.67
--log_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781864224577058-825-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 09:32:44 UTC on ccycloud.quasar-gbtldp.root.comops.site
build id 4455440
I20260619 10:17:06.598353 1207 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 10:17:06.598564 1207 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 10:17:06.603660 1213 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 10:17:06.603644 1212 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 10:17:06.603643 1215 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 10:17:06.603669 1207 server_base.cc:1034] running on GCE node
I20260619 10:17:06.604079 1207 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 10:17:06.604267 1207 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 10:17:06.605406 1207 hybrid_clock.cc:648] HybridClock initialized: now 1781864226605393 us; error 27 us; skew 500 ppm
Jun 19 10:17:06 dist-test-slave-6pt2 krb5kdc[832](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864226, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 10:17:06.608651 1207 init.cc:377] Logged in from keytab as kudu/127.0.206.67@KRBTEST.COM (short username kudu)
I20260619 10:17:06.609210 1207 webserver.cc:460] Webserver started at http://127.0.206.67:34923/ using document root <none> and password file <none>
I20260619 10:17:06.609474 1207 fs_manager.cc:359] Metadata directory not provided
I20260619 10:17:06.609534 1207 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 10:17:06.609661 1207 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 10:17:06.610953 1207 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781864224577058-825-0/minicluster-data/ts-2/data/instance:
uuid: "c9b23cdc5f384394b5daa964f078d1d0"
format_stamp: "Formatted at 2026-06-19 10:17:06 on dist-test-slave-6pt2"
server_key: "105fbb69b7391b787cbefec803c69375"
server_key_iv: "30eb8b7af7b1ba824e1fc1493e73d43a"
server_key_version: "encryptionkey@0"
I20260619 10:17:06.611428 1207 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781864224577058-825-0/minicluster-data/ts-2/wal/instance:
uuid: "c9b23cdc5f384394b5daa964f078d1d0"
format_stamp: "Formatted at 2026-06-19 10:17:06 on dist-test-slave-6pt2"
server_key: "105fbb69b7391b787cbefec803c69375"
server_key_iv: "30eb8b7af7b1ba824e1fc1493e73d43a"
server_key_version: "encryptionkey@0"
I20260619 10:17:06.613308 1207 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.000s
I20260619 10:17:06.614214 1222 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 10:17:06.614406 1207 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 10:17:06.614477 1207 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781864224577058-825-0/minicluster-data/ts-2/data,/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781864224577058-825-0/minicluster-data/ts-2/wal
uuid: "c9b23cdc5f384394b5daa964f078d1d0"
format_stamp: "Formatted at 2026-06-19 10:17:06 on dist-test-slave-6pt2"
server_key: "105fbb69b7391b787cbefec803c69375"
server_key_iv: "30eb8b7af7b1ba824e1fc1493e73d43a"
server_key_version: "encryptionkey@0"
I20260619 10:17:06.614559 1207 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781864224577058-825-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781864224577058-825-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781864224577058-825-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 10:17:06.633528 1207 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 10:17:06.635346 1207 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 10:17:06.635527 1207 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 10:17:06.635782 1207 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 10:17:06.636089 1207 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 10:17:06.636121 1207 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 10:17:06.636170 1207 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 10:17:06.636196 1207 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 10:17:06.642520 1207 rpc_server.cc:307] RPC server started. Bound to: 127.0.206.67:45901
I20260619 10:17:06.642592 1335 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.206.67:45901 every 8 connection(s)
I20260619 10:17:06.642926 1207 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781864224577058-825-0/minicluster-data/ts-2/data/info.pb
Jun 19 10:17:06 dist-test-slave-6pt2 krb5kdc[832](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864226, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.67@KRBTEST.COM for kudu/127.0.206.126@KRBTEST.COM
I20260619 10:17:06.650985 825 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTU9OP6/build/release/bin/kudu as pid 1207
I20260619 10:17:06.651085 825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781864224577058-825-0/minicluster-data/ts-2/wal/instance
I20260619 10:17:06.651268 825 external_mini_cluster.cc:1428] Setting key 3a7591439d1331525694d4e229ecb95f
I20260619 10:17:06.652163 934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:06.643629 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.67:54669 (local address 127.0.206.126:43441)
0619 10:17:06.643677 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0619 10:17:06.643680 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 10:17:06.643893 (+ 213us) server_negotiation.cc:407] Connection header received
0619 10:17:06.644349 (+ 456us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:17:06.644351 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:17:06.644361 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:17:06.644417 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:17:06.645272 (+ 855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:06.645740 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:06.646656 (+ 916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:06.646888 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:06.646957 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:06.649356 (+ 2399us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 10:17:06.649357 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 10:17:06.649358 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 10:17:06.649363 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0619 10:17:06.650644 (+ 1281us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:17:06.651057 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:17:06.651058 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:17:06.651059 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:17:06.651119 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:17:06.651407 (+ 288us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:17:06.651420 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:17:06.651420 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:17:06.651587 (+ 167us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 10:17:06.651653 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0619 10:17:06.652017 (+ 364us) server_negotiation.cc:299] Negotiation successful
0619 10:17:06.652101 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260619 10:17:06.652384 1338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:06.643626 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:43441 (local address 127.0.206.67:54669)
0619 10:17:06.643759 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0619 10:17:06.643801 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0619 10:17:06.644266 (+ 465us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 10:17:06.644467 (+ 201us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 10:17:06.644470 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 10:17:06.645010 (+ 540us) client_negotiation.cc:189] Negotiated authn=SASL
0619 10:17:06.645202 (+ 192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:06.645206 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:06.645813 (+ 607us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:06.645814 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:06.646547 (+ 733us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:06.646549 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:06.646973 (+ 424us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:06.646974 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:06.647077 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:06.647980 (+ 903us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 10:17:06.647988 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 10:17:06.649267 (+ 1279us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 10:17:06.650727 (+ 1460us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 10:17:06.650732 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 10:17:06.650734 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 10:17:06.650961 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 10:17:06.651212 (+ 251us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 10:17:06.651213 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 10:17:06.651213 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 10:17:06.651305 (+ 92us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 10:17:06.651668 (+ 363us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 10:17:06.651674 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 10:17:06.651838 (+ 164us) client_negotiation.cc:769] Sending connection context
0619 10:17:06.651971 (+ 133us) client_negotiation.cc:240] Negotiation successful
0619 10:17:06.652135 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":99,"thread_start_us":76,"threads_started":1}
I20260619 10:17:06.652824 1336 heartbeater.cc:347] Connected to a master server at 127.0.206.126:43441
I20260619 10:17:06.652897 1336 heartbeater.cc:464] Registering TS with master...
I20260619 10:17:06.653074 1336 heartbeater.cc:511] Master 127.0.206.126:43441 requested a full tablet report, sending...
I20260619 10:17:06.653425 877 ts_manager.cc:194] Registered new tserver with Master: c9b23cdc5f384394b5daa964f078d1d0 (127.0.206.67:45901)
I20260619 10:17:06.653785 877 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.206.67@KRBTEST.COM'} at 127.0.206.67:54669
I20260619 10:17:06.663376 825 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Jun 19 10:17:06 dist-test-slave-6pt2 krb5kdc[832](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864226, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Jun 19 10:17:06 dist-test-slave-6pt2 krb5kdc[832](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864226, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.0.206.65@KRBTEST.COM
I20260619 10:17:06.684861 1343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:06.677865 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57484 (local address 127.0.206.65:46657)
0619 10:17:06.678025 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0619 10:17:06.678029 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 10:17:06.678039 (+ 10us) server_negotiation.cc:407] Connection header received
0619 10:17:06.678063 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:17:06.678065 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:17:06.678091 (+ 26us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:17:06.678154 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:17:06.678706 (+ 552us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:06.679415 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:06.680063 (+ 648us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:06.680355 (+ 292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:06.680406 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:06.682450 (+ 2044us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 10:17:06.682456 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 10:17:06.682456 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 10:17:06.682462 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 10:17:06.683934 (+ 1472us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:17:06.684215 (+ 281us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:17:06.684218 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:17:06.684218 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:17:06.684266 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:17:06.684404 (+ 138us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:17:06.684405 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:17:06.684405 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:17:06.684514 (+ 109us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 10:17:06.684611 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0619 10:17:06.684663 (+ 52us) server_negotiation.cc:299] Negotiation successful
0619 10:17:06.684736 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":129,"thread_start_us":86,"threads_started":1}
W20260619 10:17:06.685256 982 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:57484
Jun 19 10:17:06 dist-test-slave-6pt2 krb5kdc[832](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864226, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Jun 19 10:17:06 dist-test-slave-6pt2 krb5kdc[832](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864226, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.206.65@KRBTEST.COM
I20260619 10:17:06.706336 1343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:06.699594 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57490 (local address 127.0.206.65:46657)
0619 10:17:06.699649 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0619 10:17:06.699651 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 10:17:06.699714 (+ 63us) server_negotiation.cc:407] Connection header received
0619 10:17:06.699760 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:17:06.699761 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:17:06.699770 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:17:06.699816 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:17:06.700305 (+ 489us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:06.700761 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:06.701414 (+ 653us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:06.701605 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:06.701672 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:06.703825 (+ 2153us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 10:17:06.703827 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 10:17:06.703827 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 10:17:06.703835 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 10:17:06.705228 (+ 1393us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:17:06.705585 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:17:06.705586 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:17:06.705587 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:17:06.705642 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:17:06.705830 (+ 188us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:17:06.705831 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:17:06.705831 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:17:06.705951 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 10:17:06.706070 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0619 10:17:06.706158 (+ 88us) server_negotiation.cc:299] Negotiation successful
0619 10:17:06.706233 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260619 10:17:06.706845 825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTU9OP6/build/release/bin/kudu with pid 939
I20260619 10:17:06.711524 825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTU9OP6/build/release/bin/kudu with pid 1073
I20260619 10:17:06.717487 825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTU9OP6/build/release/bin/kudu with pid 1207
I20260619 10:17:06.721868 825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTU9OP6/build/release/bin/kudu with pid 845
2026-06-19T10:17:06Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (2152 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781864224577058-825-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 10:17:06 dist-test-slave-6pt2 krb5kdc[1350](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 10:17:06 dist-test-slave-6pt2 krb5kdc[1350](info): set up 2 sockets
Jun 19 10:17:06 dist-test-slave-6pt2 krb5kdc[1350](info): commencing operation
krb5kdc: starting...
W20260619 10:17:07.750084 825 mini_kdc.cc:121] Time spent starting KDC: real 1.008s user 0.004s sys 0.000s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781864224577058-825-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781864224577058-825-0/krb5kdc/test-user.keytab.
Jun 19 10:17:07 dist-test-slave-6pt2 krb5kdc[1350](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864227, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T10:17:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T10:17:07Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.206.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.206.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.206.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781864224577058-825-0/krb5kdc/kudu_127.0.206.126.keytab.
Entry for principal kudu/127.0.206.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781864224577058-825-0/krb5kdc/kudu_127.0.206.126.keytab.
I20260619 10:17:07.857131 825 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTU9OP6/build/release/bin/kudu
/tmp/dist-test-taskTU9OP6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781864224577058-825-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781864224577058-825-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781864224577058-825-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781864224577058-825-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.206.126:36805
--webserver_interface=127.0.206.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781864224577058-825-0/krb5kdc/kudu_127.0.206.126.keytab
--principal=kudu/127.0.206.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.206.84:33721
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.206.126:36805
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781864224577058-825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781864224577058-825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781864224577058-825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 10:17:07.935573 1363 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 10:17:07.935818 1363 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 10:17:07.935854 1363 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 10:17:07.937001 1363 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 10:17:07.937074 1363 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 10:17:07.937098 1363 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 10:17:07.937119 1363 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 10:17:07.937150 1363 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 10:17:07.938565 1363 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.206.84:33721
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781864224577058-825-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781864224577058-825-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.206.126:36805
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781864224577058-825-0/krb5kdc/kudu_127.0.206.126.keytab
--principal=kudu/127.0.206.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.206.126:36805
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781864224577058-825-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.206.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1363
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781864224577058-825-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 09:32:44 UTC on ccycloud.quasar-gbtldp.root.comops.site
build id 4455440
I20260619 10:17:07.938859 1363 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 10:17:07.939074 1363 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 10:17:07.943859 1371 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 10:17:07.943828 1369 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 10:17:07.943833 1368 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 10:17:07.944046 1363 server_base.cc:1034] running on GCE node
I20260619 10:17:07.944288 1363 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 10:17:07.944589 1363 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 10:17:07.945725 1363 hybrid_clock.cc:648] HybridClock initialized: now 1781864227945713 us; error 33 us; skew 500 ppm
Jun 19 10:17:07 dist-test-slave-6pt2 krb5kdc[1350](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864227, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 10:17:07.948168 1363 init.cc:377] Logged in from keytab as kudu/127.0.206.126@KRBTEST.COM (short username kudu)
I20260619 10:17:07.948572 1363 webserver.cc:460] Webserver started at http://127.0.206.126:39729/ using document root <none> and password file <none>
I20260619 10:17:07.948784 1363 fs_manager.cc:359] Metadata directory not provided
I20260619 10:17:07.948827 1363 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 10:17:07.948966 1363 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 10:17:07.949967 1363 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781864224577058-825-0/minicluster-data/master-0/data/instance:
uuid: "6d20516fb2ff4f15947699361a12cb0b"
format_stamp: "Formatted at 2026-06-19 10:17:07 on dist-test-slave-6pt2"
server_key: "f3ec9c07e59fd64d9fce57da7b64e890"
server_key_iv: "6468507d905b6b429dc16ac8dc5c44f5"
server_key_version: "encryptionkey@0"
I20260619 10:17:07.950309 1363 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781864224577058-825-0/minicluster-data/master-0/wal/instance:
uuid: "6d20516fb2ff4f15947699361a12cb0b"
format_stamp: "Formatted at 2026-06-19 10:17:07 on dist-test-slave-6pt2"
server_key: "f3ec9c07e59fd64d9fce57da7b64e890"
server_key_iv: "6468507d905b6b429dc16ac8dc5c44f5"
server_key_version: "encryptionkey@0"
I20260619 10:17:07.951718 1363 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 10:17:07.952428 1378 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 10:17:07.952595 1363 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 10:17:07.952695 1363 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781864224577058-825-0/minicluster-data/master-0/data,/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781864224577058-825-0/minicluster-data/master-0/wal
uuid: "6d20516fb2ff4f15947699361a12cb0b"
format_stamp: "Formatted at 2026-06-19 10:17:07 on dist-test-slave-6pt2"
server_key: "f3ec9c07e59fd64d9fce57da7b64e890"
server_key_iv: "6468507d905b6b429dc16ac8dc5c44f5"
server_key_version: "encryptionkey@0"
I20260619 10:17:07.952767 1363 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781864224577058-825-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781864224577058-825-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781864224577058-825-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 10:17:07.975657 1363 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 10:17:07.977929 1363 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 10:17:07.978093 1363 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 10:17:07.981866 1363 rpc_server.cc:307] RPC server started. Bound to: 127.0.206.126:36805
I20260619 10:17:07.981913 1430 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.206.126:36805 every 8 connection(s)
I20260619 10:17:07.982275 1363 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781864224577058-825-0/minicluster-data/master-0/data/info.pb
I20260619 10:17:07.983191 1431 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 10:17:07.985536 1431 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6d20516fb2ff4f15947699361a12cb0b: Bootstrap starting.
I20260619 10:17:07.986092 1431 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6d20516fb2ff4f15947699361a12cb0b: Neither blocks nor log segments found. Creating new log.
I20260619 10:17:07.986399 1431 log.cc:824] T 00000000000000000000000000000000 P 6d20516fb2ff4f15947699361a12cb0b: Log is configured to *not* fsync() on all Append() calls
I20260619 10:17:07.987102 1431 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6d20516fb2ff4f15947699361a12cb0b: No bootstrap required, opened a new log
I20260619 10:17:07.988797 1431 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6d20516fb2ff4f15947699361a12cb0b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d20516fb2ff4f15947699361a12cb0b" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 36805 } }
I20260619 10:17:07.988930 1431 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6d20516fb2ff4f15947699361a12cb0b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 10:17:07.988953 1431 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6d20516fb2ff4f15947699361a12cb0b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6d20516fb2ff4f15947699361a12cb0b, State: Initialized, Role: FOLLOWER
I20260619 10:17:07.989128 1431 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6d20516fb2ff4f15947699361a12cb0b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d20516fb2ff4f15947699361a12cb0b" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 36805 } }
I20260619 10:17:07.989193 1431 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6d20516fb2ff4f15947699361a12cb0b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 10:17:07.989234 1431 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6d20516fb2ff4f15947699361a12cb0b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 10:17:07.989281 1431 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6d20516fb2ff4f15947699361a12cb0b [term 0 FOLLOWER]: Advancing to term 1
I20260619 10:17:07.989768 1431 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6d20516fb2ff4f15947699361a12cb0b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d20516fb2ff4f15947699361a12cb0b" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 36805 } }
I20260619 10:17:07.989876 1431 leader_election.cc:302] T 00000000000000000000000000000000 P 6d20516fb2ff4f15947699361a12cb0b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6d20516fb2ff4f15947699361a12cb0b; no voters:
I20260619 10:17:07.990053 1431 leader_election.cc:288] T 00000000000000000000000000000000 P 6d20516fb2ff4f15947699361a12cb0b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 10:17:07.990115 1434 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6d20516fb2ff4f15947699361a12cb0b [term 1 FOLLOWER]: Leader election won for term 1
I20260619 10:17:07.990286 1431 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6d20516fb2ff4f15947699361a12cb0b [sys.catalog]: configured and running, proceeding with master startup.
I20260619 10:17:07.990451 1434 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6d20516fb2ff4f15947699361a12cb0b [term 1 LEADER]: Becoming Leader. State: Replica: 6d20516fb2ff4f15947699361a12cb0b, State: Running, Role: LEADER
I20260619 10:17:07.990546 1434 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6d20516fb2ff4f15947699361a12cb0b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d20516fb2ff4f15947699361a12cb0b" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 36805 } }
I20260619 10:17:07.990907 1435 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6d20516fb2ff4f15947699361a12cb0b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6d20516fb2ff4f15947699361a12cb0b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d20516fb2ff4f15947699361a12cb0b" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 36805 } } }
I20260619 10:17:07.990916 1436 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6d20516fb2ff4f15947699361a12cb0b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6d20516fb2ff4f15947699361a12cb0b. Latest consensus state: current_term: 1 leader_uuid: "6d20516fb2ff4f15947699361a12cb0b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d20516fb2ff4f15947699361a12cb0b" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 36805 } } }
I20260619 10:17:07.991011 1435 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6d20516fb2ff4f15947699361a12cb0b [sys.catalog]: This master's current role is: LEADER
I20260619 10:17:07.991015 1436 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6d20516fb2ff4f15947699361a12cb0b [sys.catalog]: This master's current role is: LEADER
I20260619 10:17:07.991261 1443 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 10:17:07.991576 825 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTU9OP6/build/release/bin/kudu as pid 1363
I20260619 10:17:07.991681 825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781864224577058-825-0/minicluster-data/master-0/wal/instance
I20260619 10:17:07.991833 1443 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 10:17:07.991873 825 external_mini_cluster.cc:1428] Setting key d9c6b62dcfb5fc67b5e47df0514ec2ba
I20260619 10:17:07.993472 1443 catalog_manager.cc:1347] Generated new cluster ID: cbdceaca411b423bbefa7b01d178952d
I20260619 10:17:07.993520 1443 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Jun 19 10:17:07 dist-test-slave-6pt2 krb5kdc[1350](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864227, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.206.126@KRBTEST.COM
I20260619 10:17:08.003391 1453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:07.992289 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51646 (local address 127.0.206.126:36805)
0619 10:17:07.992491 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0619 10:17:07.992500 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 10:17:07.992699 (+ 199us) server_negotiation.cc:407] Connection header received
0619 10:17:07.992818 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:17:07.992824 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:17:07.992875 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:17:07.993017 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:17:07.993731 (+ 714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:07.995133 (+ 1402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:07.996209 (+ 1076us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:07.996841 (+ 632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:07.996931 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:07.999726 (+ 2795us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 10:17:07.999730 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 10:17:07.999731 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 10:17:07.999742 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0619 10:17:08.001681 (+ 1939us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:17:08.002051 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:17:08.002054 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:17:08.002055 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:17:08.002158 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:17:08.002350 (+ 192us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:17:08.002351 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:17:08.002351 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:17:08.002690 (+ 339us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 10:17:08.002836 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0619 10:17:08.002930 (+ 94us) server_negotiation.cc:299] Negotiation successful
0619 10:17:08.003167 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":161,"thread_start_us":78,"threads_started":1}
I20260619 10:17:08.014226 1443 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 10:17:08.014760 1443 catalog_manager.cc:1504] Loading token signing keys...
I20260619 10:17:08.019300 1443 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6d20516fb2ff4f15947699361a12cb0b: Generated new TSK 0
I20260619 10:17:08.019464 1443 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.0.206.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.206.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.206.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781864224577058-825-0/krb5kdc/kudu_127.0.206.65.keytab.
Entry for principal kudu/127.0.206.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781864224577058-825-0/krb5kdc/kudu_127.0.206.65.keytab.
I20260619 10:17:08.046964 825 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTU9OP6/build/release/bin/kudu
/tmp/dist-test-taskTU9OP6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781864224577058-825-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781864224577058-825-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781864224577058-825-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781864224577058-825-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.206.65:0
--local_ip_for_outbound_sockets=127.0.206.65
--webserver_interface=127.0.206.65
--webserver_port=0
--tserver_master_addrs=127.0.206.126:36805
--keytab_file=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781864224577058-825-0/krb5kdc/kudu_127.0.206.65.keytab
--principal=kudu/127.0.206.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.206.84:33721
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781864224577058-825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781864224577058-825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781864224577058-825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 10:17:08.132059 1457 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 10:17:08.132288 1457 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 10:17:08.132324 1457 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 10:17:08.133491 1457 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 10:17:08.133567 1457 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 10:17:08.133610 1457 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.206.65
I20260619 10:17:08.135007 1457 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.206.84:33721
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781864224577058-825-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781864224577058-825-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781864224577058-825-0/krb5kdc/kudu_127.0.206.65.keytab
--principal=kudu/127.0.206.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.206.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781864224577058-825-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.206.65
--webserver_port=0
--tserver_master_addrs=127.0.206.126:36805
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1457
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.206.65
--log_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781864224577058-825-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 09:32:44 UTC on ccycloud.quasar-gbtldp.root.comops.site
build id 4455440
I20260619 10:17:08.135250 1457 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 10:17:08.135454 1457 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 10:17:08.140220 1463 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 10:17:08.140237 1462 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 10:17:08.140264 1465 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 10:17:08.140532 1457 server_base.cc:1034] running on GCE node
I20260619 10:17:08.140763 1457 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 10:17:08.141013 1457 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 10:17:08.142160 1457 hybrid_clock.cc:648] HybridClock initialized: now 1781864228142145 us; error 32 us; skew 500 ppm
Jun 19 10:17:08 dist-test-slave-6pt2 krb5kdc[1350](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864228, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 10:17:08.144623 1457 init.cc:377] Logged in from keytab as kudu/127.0.206.65@KRBTEST.COM (short username kudu)
I20260619 10:17:08.145046 1457 webserver.cc:460] Webserver started at http://127.0.206.65:34655/ using document root <none> and password file <none>
I20260619 10:17:08.145289 1457 fs_manager.cc:359] Metadata directory not provided
I20260619 10:17:08.145357 1457 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 10:17:08.145500 1457 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 10:17:08.146498 1457 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781864224577058-825-0/minicluster-data/ts-0/data/instance:
uuid: "ef84a4869075487cba970c5831c8a912"
format_stamp: "Formatted at 2026-06-19 10:17:08 on dist-test-slave-6pt2"
server_key: "490a835bc05cb6b771235a053f0edbd1"
server_key_iv: "9e85fe201c41946a1411b0123aff1459"
server_key_version: "encryptionkey@0"
I20260619 10:17:08.146907 1457 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781864224577058-825-0/minicluster-data/ts-0/wal/instance:
uuid: "ef84a4869075487cba970c5831c8a912"
format_stamp: "Formatted at 2026-06-19 10:17:08 on dist-test-slave-6pt2"
server_key: "490a835bc05cb6b771235a053f0edbd1"
server_key_iv: "9e85fe201c41946a1411b0123aff1459"
server_key_version: "encryptionkey@0"
I20260619 10:17:08.148202 1457 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 10:17:08.148986 1472 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 10:17:08.149178 1457 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 10:17:08.149276 1457 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781864224577058-825-0/minicluster-data/ts-0/data,/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781864224577058-825-0/minicluster-data/ts-0/wal
uuid: "ef84a4869075487cba970c5831c8a912"
format_stamp: "Formatted at 2026-06-19 10:17:08 on dist-test-slave-6pt2"
server_key: "490a835bc05cb6b771235a053f0edbd1"
server_key_iv: "9e85fe201c41946a1411b0123aff1459"
server_key_version: "encryptionkey@0"
I20260619 10:17:08.149359 1457 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781864224577058-825-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781864224577058-825-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781864224577058-825-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 10:17:08.164995 1457 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 10:17:08.166704 1457 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 10:17:08.166862 1457 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 10:17:08.167065 1457 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 10:17:08.167356 1457 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 10:17:08.167409 1457 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 10:17:08.167449 1457 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 10:17:08.167477 1457 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 10:17:08.173089 1457 rpc_server.cc:307] RPC server started. Bound to: 127.0.206.65:42851
I20260619 10:17:08.173126 1585 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.206.65:42851 every 8 connection(s)
I20260619 10:17:08.173444 1457 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781864224577058-825-0/minicluster-data/ts-0/data/info.pb
Jun 19 10:17:08 dist-test-slave-6pt2 krb5kdc[1350](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864228, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.65@KRBTEST.COM for kudu/127.0.206.126@KRBTEST.COM
I20260619 10:17:08.181629 1453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:08.174145 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.65:50359 (local address 127.0.206.126:36805)
0619 10:17:08.174222 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0619 10:17:08.174224 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 10:17:08.174368 (+ 144us) server_negotiation.cc:407] Connection header received
0619 10:17:08.174792 (+ 424us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:17:08.174792 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:17:08.174805 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:17:08.174849 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:17:08.175658 (+ 809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:08.176118 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:08.176999 (+ 881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:08.177201 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:08.177271 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:08.179360 (+ 2089us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 10:17:08.179361 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 10:17:08.179361 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 10:17:08.179369 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 10:17:08.180562 (+ 1193us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:17:08.180891 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:17:08.180892 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:17:08.180892 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:17:08.180938 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:17:08.181145 (+ 207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:17:08.181146 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:17:08.181146 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:17:08.181239 (+ 93us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 10:17:08.181288 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0619 10:17:08.181496 (+ 208us) server_negotiation.cc:299] Negotiation successful
0619 10:17:08.181566 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260619 10:17:08.181685 825 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTU9OP6/build/release/bin/kudu as pid 1457
I20260619 10:17:08.181772 825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781864224577058-825-0/minicluster-data/ts-0/wal/instance
I20260619 10:17:08.181919 825 external_mini_cluster.cc:1428] Setting key 6320a971ea769c9d5b09702f1524f1fb
I20260619 10:17:08.181948 1588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:08.174103 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:36805 (local address 127.0.206.65:50359)
0619 10:17:08.174252 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0619 10:17:08.174283 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0619 10:17:08.174707 (+ 424us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 10:17:08.174905 (+ 198us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 10:17:08.174908 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 10:17:08.175415 (+ 507us) client_negotiation.cc:189] Negotiated authn=SASL
0619 10:17:08.175582 (+ 167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:08.175586 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:08.176191 (+ 605us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:08.176192 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:08.176929 (+ 737us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:08.176930 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:08.177298 (+ 368us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:08.177299 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:08.177396 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:08.177885 (+ 489us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 10:17:08.177893 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 10:17:08.179281 (+ 1388us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 10:17:08.180629 (+ 1348us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 10:17:08.180634 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 10:17:08.180636 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 10:17:08.180828 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 10:17:08.180993 (+ 165us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 10:17:08.180993 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 10:17:08.180994 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 10:17:08.181093 (+ 99us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 10:17:08.181291 (+ 198us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 10:17:08.181295 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 10:17:08.181394 (+ 99us) client_negotiation.cc:769] Sending connection context
0619 10:17:08.181470 (+ 76us) client_negotiation.cc:240] Negotiation successful
0619 10:17:08.181660 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":96,"threads_started":1}
I20260619 10:17:08.182458 1586 heartbeater.cc:347] Connected to a master server at 127.0.206.126:36805
I20260619 10:17:08.182551 1586 heartbeater.cc:464] Registering TS with master...
I20260619 10:17:08.182744 1586 heartbeater.cc:511] Master 127.0.206.126:36805 requested a full tablet report, sending...
I20260619 10:17:08.183244 1395 ts_manager.cc:194] Registered new tserver with Master: ef84a4869075487cba970c5831c8a912 (127.0.206.65:42851)
I20260619 10:17:08.183851 1395 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.206.65@KRBTEST.COM'} at 127.0.206.65:50359
WARNING: no policy specified for kudu/127.0.206.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.206.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.206.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781864224577058-825-0/krb5kdc/kudu_127.0.206.66.keytab.
Entry for principal kudu/127.0.206.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781864224577058-825-0/krb5kdc/kudu_127.0.206.66.keytab.
I20260619 10:17:08.203341 825 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTU9OP6/build/release/bin/kudu
/tmp/dist-test-taskTU9OP6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781864224577058-825-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781864224577058-825-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781864224577058-825-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781864224577058-825-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.206.66:0
--local_ip_for_outbound_sockets=127.0.206.66
--webserver_interface=127.0.206.66
--webserver_port=0
--tserver_master_addrs=127.0.206.126:36805
--keytab_file=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781864224577058-825-0/krb5kdc/kudu_127.0.206.66.keytab
--principal=kudu/127.0.206.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.206.84:33721
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781864224577058-825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781864224577058-825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781864224577058-825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 10:17:08.281091 1591 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 10:17:08.281317 1591 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 10:17:08.281355 1591 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 10:17:08.282506 1591 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 10:17:08.282572 1591 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 10:17:08.282635 1591 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.206.66
I20260619 10:17:08.284090 1591 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.206.84:33721
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781864224577058-825-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781864224577058-825-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781864224577058-825-0/krb5kdc/kudu_127.0.206.66.keytab
--principal=kudu/127.0.206.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.206.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781864224577058-825-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.206.66
--webserver_port=0
--tserver_master_addrs=127.0.206.126:36805
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1591
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.206.66
--log_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781864224577058-825-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 09:32:44 UTC on ccycloud.quasar-gbtldp.root.comops.site
build id 4455440
I20260619 10:17:08.284348 1591 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 10:17:08.284569 1591 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 10:17:08.289381 1599 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 10:17:08.289407 1597 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 10:17:08.289459 1591 server_base.cc:1034] running on GCE node
W20260619 10:17:08.289414 1596 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 10:17:08.289820 1591 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 10:17:08.290102 1591 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 10:17:08.291257 1591 hybrid_clock.cc:648] HybridClock initialized: now 1781864228291246 us; error 29 us; skew 500 ppm
Jun 19 10:17:08 dist-test-slave-6pt2 krb5kdc[1350](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864228, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 10:17:08.293712 1591 init.cc:377] Logged in from keytab as kudu/127.0.206.66@KRBTEST.COM (short username kudu)
I20260619 10:17:08.294121 1591 webserver.cc:460] Webserver started at http://127.0.206.66:41867/ using document root <none> and password file <none>
I20260619 10:17:08.294337 1591 fs_manager.cc:359] Metadata directory not provided
I20260619 10:17:08.294378 1591 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 10:17:08.294513 1591 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 10:17:08.295538 1591 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781864224577058-825-0/minicluster-data/ts-1/data/instance:
uuid: "c3691603b6c044f8aca1a41fec1bc69e"
format_stamp: "Formatted at 2026-06-19 10:17:08 on dist-test-slave-6pt2"
server_key: "a74a7e0e70b9c48957259d85a928b8ea"
server_key_iv: "6642caf4d52bb61f34c4e6a1212e66ee"
server_key_version: "encryptionkey@0"
I20260619 10:17:08.295884 1591 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781864224577058-825-0/minicluster-data/ts-1/wal/instance:
uuid: "c3691603b6c044f8aca1a41fec1bc69e"
format_stamp: "Formatted at 2026-06-19 10:17:08 on dist-test-slave-6pt2"
server_key: "a74a7e0e70b9c48957259d85a928b8ea"
server_key_iv: "6642caf4d52bb61f34c4e6a1212e66ee"
server_key_version: "encryptionkey@0"
I20260619 10:17:08.297187 1591 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260619 10:17:08.297845 1606 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 10:17:08.297991 1591 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260619 10:17:08.298075 1591 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781864224577058-825-0/minicluster-data/ts-1/data,/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781864224577058-825-0/minicluster-data/ts-1/wal
uuid: "c3691603b6c044f8aca1a41fec1bc69e"
format_stamp: "Formatted at 2026-06-19 10:17:08 on dist-test-slave-6pt2"
server_key: "a74a7e0e70b9c48957259d85a928b8ea"
server_key_iv: "6642caf4d52bb61f34c4e6a1212e66ee"
server_key_version: "encryptionkey@0"
I20260619 10:17:08.298126 1591 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781864224577058-825-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781864224577058-825-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781864224577058-825-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 10:17:08.308959 1591 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 10:17:08.311069 1591 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 10:17:08.311178 1591 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 10:17:08.311337 1591 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 10:17:08.311558 1591 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 10:17:08.311589 1591 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 10:17:08.311609 1591 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 10:17:08.311621 1591 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 10:17:08.316937 1591 rpc_server.cc:307] RPC server started. Bound to: 127.0.206.66:42881
I20260619 10:17:08.317011 1719 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.206.66:42881 every 8 connection(s)
I20260619 10:17:08.317294 1591 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781864224577058-825-0/minicluster-data/ts-1/data/info.pb
Jun 19 10:17:08 dist-test-slave-6pt2 krb5kdc[1350](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864228, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.66@KRBTEST.COM for kudu/127.0.206.126@KRBTEST.COM
I20260619 10:17:08.325436 1453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:08.317925 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.66:54855 (local address 127.0.206.126:36805)
0619 10:17:08.317995 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0619 10:17:08.317997 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 10:17:08.318175 (+ 178us) server_negotiation.cc:407] Connection header received
0619 10:17:08.318741 (+ 566us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:17:08.318742 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:17:08.318752 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:17:08.318815 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:17:08.319670 (+ 855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:08.320150 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:08.321014 (+ 864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:08.321192 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:08.321239 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:08.323236 (+ 1997us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 10:17:08.323237 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 10:17:08.323237 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 10:17:08.323243 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 10:17:08.324359 (+ 1116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:17:08.324689 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:17:08.324690 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:17:08.324690 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:17:08.324737 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:17:08.324928 (+ 191us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:17:08.324928 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:17:08.324929 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:17:08.325021 (+ 92us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 10:17:08.325068 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0619 10:17:08.325280 (+ 212us) server_negotiation.cc:299] Negotiation successful
0619 10:17:08.325344 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260619 10:17:08.325543 1722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:08.317880 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:36805 (local address 127.0.206.66:54855)
0619 10:17:08.318033 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0619 10:17:08.318072 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0619 10:17:08.318522 (+ 450us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 10:17:08.318891 (+ 369us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 10:17:08.318895 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 10:17:08.319423 (+ 528us) client_negotiation.cc:189] Negotiated authn=SASL
0619 10:17:08.319597 (+ 174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:08.319601 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:08.320222 (+ 621us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:08.320223 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:08.320945 (+ 722us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:08.320947 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:08.321256 (+ 309us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:08.321257 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:08.321346 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:08.321756 (+ 410us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 10:17:08.321764 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 10:17:08.323146 (+ 1382us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 10:17:08.324424 (+ 1278us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 10:17:08.324429 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 10:17:08.324431 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 10:17:08.324627 (+ 196us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 10:17:08.324790 (+ 163us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 10:17:08.324791 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 10:17:08.324791 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 10:17:08.324877 (+ 86us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 10:17:08.325072 (+ 195us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 10:17:08.325081 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 10:17:08.325179 (+ 98us) client_negotiation.cc:769] Sending connection context
0619 10:17:08.325255 (+ 76us) client_negotiation.cc:240] Negotiation successful
0619 10:17:08.325362 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":86,"threads_started":1}
I20260619 10:17:08.325958 1720 heartbeater.cc:347] Connected to a master server at 127.0.206.126:36805
I20260619 10:17:08.326032 1720 heartbeater.cc:464] Registering TS with master...
I20260619 10:17:08.326154 1720 heartbeater.cc:511] Master 127.0.206.126:36805 requested a full tablet report, sending...
I20260619 10:17:08.326437 1395 ts_manager.cc:194] Registered new tserver with Master: c3691603b6c044f8aca1a41fec1bc69e (127.0.206.66:42881)
I20260619 10:17:08.326850 1395 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.206.66@KRBTEST.COM'} at 127.0.206.66:54855
I20260619 10:17:08.327356 825 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTU9OP6/build/release/bin/kudu as pid 1591
I20260619 10:17:08.327443 825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781864224577058-825-0/minicluster-data/ts-1/wal/instance
I20260619 10:17:08.327602 825 external_mini_cluster.cc:1428] Setting key 8d6054245a93eea37d0fb7af830292c0
WARNING: no policy specified for kudu/127.0.206.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.206.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.206.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781864224577058-825-0/krb5kdc/kudu_127.0.206.67.keytab.
Entry for principal kudu/127.0.206.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781864224577058-825-0/krb5kdc/kudu_127.0.206.67.keytab.
I20260619 10:17:08.349278 825 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTU9OP6/build/release/bin/kudu
/tmp/dist-test-taskTU9OP6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781864224577058-825-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781864224577058-825-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781864224577058-825-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781864224577058-825-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.206.67:0
--local_ip_for_outbound_sockets=127.0.206.67
--webserver_interface=127.0.206.67
--webserver_port=0
--tserver_master_addrs=127.0.206.126:36805
--keytab_file=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781864224577058-825-0/krb5kdc/kudu_127.0.206.67.keytab
--principal=kudu/127.0.206.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.206.84:33721
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781864224577058-825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781864224577058-825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781864224577058-825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 10:17:08.426973 1725 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 10:17:08.427151 1725 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 10:17:08.427174 1725 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 10:17:08.428309 1725 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 10:17:08.428350 1725 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 10:17:08.428403 1725 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.206.67
I20260619 10:17:08.429777 1725 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.206.84:33721
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781864224577058-825-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781864224577058-825-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781864224577058-825-0/krb5kdc/kudu_127.0.206.67.keytab
--principal=kudu/127.0.206.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.206.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781864224577058-825-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.206.67
--webserver_port=0
--tserver_master_addrs=127.0.206.126:36805
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1725
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.206.67
--log_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781864224577058-825-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 09:32:44 UTC on ccycloud.quasar-gbtldp.root.comops.site
build id 4455440
I20260619 10:17:08.429939 1725 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 10:17:08.430178 1725 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 10:17:08.434937 1731 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 10:17:08.434950 1730 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 10:17:08.435045 1725 server_base.cc:1034] running on GCE node
W20260619 10:17:08.434953 1733 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 10:17:08.435480 1725 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 10:17:08.435729 1725 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 10:17:08.436888 1725 hybrid_clock.cc:648] HybridClock initialized: now 1781864228436880 us; error 38 us; skew 500 ppm
Jun 19 10:17:08 dist-test-slave-6pt2 krb5kdc[1350](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864228, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 10:17:08.439227 1725 init.cc:377] Logged in from keytab as kudu/127.0.206.67@KRBTEST.COM (short username kudu)
I20260619 10:17:08.439661 1725 webserver.cc:460] Webserver started at http://127.0.206.67:35871/ using document root <none> and password file <none>
I20260619 10:17:08.439891 1725 fs_manager.cc:359] Metadata directory not provided
I20260619 10:17:08.439958 1725 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 10:17:08.440109 1725 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 10:17:08.441071 1725 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781864224577058-825-0/minicluster-data/ts-2/data/instance:
uuid: "eb2a6db87881499c8c5301b38ae8d850"
format_stamp: "Formatted at 2026-06-19 10:17:08 on dist-test-slave-6pt2"
server_key: "07c42fc88275e6fc4678527ac83f53d5"
server_key_iv: "4764370c1d104daeecae2951ad76798e"
server_key_version: "encryptionkey@0"
I20260619 10:17:08.441406 1725 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781864224577058-825-0/minicluster-data/ts-2/wal/instance:
uuid: "eb2a6db87881499c8c5301b38ae8d850"
format_stamp: "Formatted at 2026-06-19 10:17:08 on dist-test-slave-6pt2"
server_key: "07c42fc88275e6fc4678527ac83f53d5"
server_key_iv: "4764370c1d104daeecae2951ad76798e"
server_key_version: "encryptionkey@0"
I20260619 10:17:08.442648 1725 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260619 10:17:08.443327 1740 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 10:17:08.443495 1725 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260619 10:17:08.443579 1725 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781864224577058-825-0/minicluster-data/ts-2/data,/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781864224577058-825-0/minicluster-data/ts-2/wal
uuid: "eb2a6db87881499c8c5301b38ae8d850"
format_stamp: "Formatted at 2026-06-19 10:17:08 on dist-test-slave-6pt2"
server_key: "07c42fc88275e6fc4678527ac83f53d5"
server_key_iv: "4764370c1d104daeecae2951ad76798e"
server_key_version: "encryptionkey@0"
I20260619 10:17:08.443662 1725 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781864224577058-825-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781864224577058-825-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781864224577058-825-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 10:17:08.468987 1725 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 10:17:08.470809 1725 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 10:17:08.470963 1725 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 10:17:08.471194 1725 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 10:17:08.471486 1725 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 10:17:08.471539 1725 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 10:17:08.471575 1725 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 10:17:08.471601 1725 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 10:17:08.477139 1725 rpc_server.cc:307] RPC server started. Bound to: 127.0.206.67:37739
I20260619 10:17:08.477188 1853 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.206.67:37739 every 8 connection(s)
I20260619 10:17:08.477496 1725 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781864224577058-825-0/minicluster-data/ts-2/data/info.pb
Jun 19 10:17:08 dist-test-slave-6pt2 krb5kdc[1350](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864228, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.67@KRBTEST.COM for kudu/127.0.206.126@KRBTEST.COM
I20260619 10:17:08.483590 825 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTU9OP6/build/release/bin/kudu as pid 1725
I20260619 10:17:08.483696 825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781864224577058-825-0/minicluster-data/ts-2/wal/instance
I20260619 10:17:08.483858 825 external_mini_cluster.cc:1428] Setting key 2dee05e2a85fccd66c527850e21579ff
I20260619 10:17:08.486246 1453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:08.478184 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.67:47103 (local address 127.0.206.126:36805)
0619 10:17:08.478244 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0619 10:17:08.478246 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 10:17:08.478417 (+ 171us) server_negotiation.cc:407] Connection header received
0619 10:17:08.478983 (+ 566us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:17:08.478985 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:17:08.478997 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:17:08.479070 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:17:08.479935 (+ 865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:08.480415 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:08.481327 (+ 912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:08.481496 (+ 169us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:08.481545 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:08.483482 (+ 1937us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 10:17:08.483484 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 10:17:08.483485 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 10:17:08.483492 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 10:17:08.484923 (+ 1431us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:17:08.485327 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:17:08.485328 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:17:08.485328 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:17:08.485387 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:17:08.485631 (+ 244us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:17:08.485632 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:17:08.485633 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:17:08.485748 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 10:17:08.485824 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0619 10:17:08.486081 (+ 257us) server_negotiation.cc:299] Negotiation successful
0619 10:17:08.486154 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260619 10:17:08.486433 1856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:08.478154 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:36805 (local address 127.0.206.67:47103)
0619 10:17:08.478299 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0619 10:17:08.478339 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0619 10:17:08.478881 (+ 542us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 10:17:08.479120 (+ 239us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 10:17:08.479123 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 10:17:08.479669 (+ 546us) client_negotiation.cc:189] Negotiated authn=SASL
0619 10:17:08.479858 (+ 189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:08.479862 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:08.480491 (+ 629us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:08.480491 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:08.481254 (+ 763us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:08.481255 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:08.481571 (+ 316us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:08.481572 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:08.481672 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:08.482057 (+ 385us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 10:17:08.482065 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 10:17:08.483370 (+ 1305us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 10:17:08.485025 (+ 1655us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 10:17:08.485031 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 10:17:08.485033 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 10:17:08.485250 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 10:17:08.485453 (+ 203us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 10:17:08.485454 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 10:17:08.485454 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 10:17:08.485563 (+ 109us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 10:17:08.485815 (+ 252us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 10:17:08.485821 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 10:17:08.485953 (+ 132us) client_negotiation.cc:769] Sending connection context
0619 10:17:08.486058 (+ 105us) client_negotiation.cc:240] Negotiation successful
0619 10:17:08.486216 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":81,"threads_started":1}
I20260619 10:17:08.486928 1854 heartbeater.cc:347] Connected to a master server at 127.0.206.126:36805
I20260619 10:17:08.487037 1854 heartbeater.cc:464] Registering TS with master...
I20260619 10:17:08.487207 1854 heartbeater.cc:511] Master 127.0.206.126:36805 requested a full tablet report, sending...
I20260619 10:17:08.487571 1395 ts_manager.cc:194] Registered new tserver with Master: eb2a6db87881499c8c5301b38ae8d850 (127.0.206.67:37739)
I20260619 10:17:08.487926 1395 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.206.67@KRBTEST.COM'} at 127.0.206.67:47103
I20260619 10:17:08.495502 825 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 10:17:08.502316 1453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:08.496984 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51654 (local address 127.0.206.126:36805)
0619 10:17:08.497033 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0619 10:17:08.497035 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 10:17:08.497130 (+ 95us) server_negotiation.cc:407] Connection header received
0619 10:17:08.497165 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:17:08.497165 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:17:08.497172 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:17:08.497219 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:17:08.497741 (+ 522us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:08.498180 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:08.498890 (+ 710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:08.499058 (+ 168us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:08.499116 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:08.500114 (+ 998us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 10:17:08.500115 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 10:17:08.500115 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 10:17:08.500120 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0619 10:17:08.501417 (+ 1297us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:17:08.501755 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:17:08.501756 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:17:08.501756 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:17:08.501803 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:17:08.501954 (+ 151us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:17:08.501955 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:17:08.501955 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:17:08.502061 (+ 106us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 10:17:08.502139 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0619 10:17:08.502209 (+ 70us) server_negotiation.cc:299] Negotiation successful
0619 10:17:08.502261 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260619 10:17:08.504261 1395 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51654:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 10:17:08.505677 1395 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 10:17:08.511861 1867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:08.508061 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43800 (local address 127.0.206.66:42881)
0619 10:17:08.508246 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0619 10:17:08.508250 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 10:17:08.508259 (+ 9us) server_negotiation.cc:407] Connection header received
0619 10:17:08.508294 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:17:08.508296 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:17:08.508312 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:17:08.508384 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 10:17:08.508636 (+ 252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:08.509667 (+ 1031us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:08.510656 (+ 989us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:08.511388 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:08.511447 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:08.511500 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0619 10:17:08.511737 (+ 237us) server_negotiation.cc:299] Negotiation successful
0619 10:17:08.511778 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":156,"thread_start_us":73,"threads_started":1}
I20260619 10:17:08.511890 1862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:08.507675 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.67:37739 (local address 127.0.0.1:51468)
0619 10:17:08.507778 (+ 103us) negotiation.cc:107] Waiting for socket to connect
0619 10:17:08.507814 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0619 10:17:08.507870 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 10:17:08.508226 (+ 356us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 10:17:08.508229 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 10:17:08.508231 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 10:17:08.508290 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:08.508291 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:08.509523 (+ 1232us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:08.509524 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:08.510869 (+ 1345us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:08.510870 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:08.511513 (+ 643us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:08.511514 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:08.511601 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:08.511613 (+ 12us) client_negotiation.cc:769] Sending connection context
0619 10:17:08.511725 (+ 112us) client_negotiation.cc:240] Negotiation successful
0619 10:17:08.511767 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":81,"mutex_wait_us":48,"thread_start_us":61,"threads_started":1}
I20260619 10:17:08.512049 1865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:08.507866 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.66:42881 (local address 127.0.0.1:43800)
0619 10:17:08.508040 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0619 10:17:08.508051 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 10:17:08.508098 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 10:17:08.508458 (+ 360us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 10:17:08.508459 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 10:17:08.508461 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 10:17:08.508503 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:08.508505 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:08.509764 (+ 1259us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:08.509765 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:08.510547 (+ 782us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:08.510550 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:08.511539 (+ 989us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:08.511540 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:08.511606 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:08.511625 (+ 19us) client_negotiation.cc:769] Sending connection context
0619 10:17:08.511686 (+ 61us) client_negotiation.cc:240] Negotiation successful
0619 10:17:08.511734 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":161,"thread_start_us":52,"threads_started":1}
I20260619 10:17:08.512061 1866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:08.507813 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52288 (local address 127.0.206.65:42851)
0619 10:17:08.507989 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0619 10:17:08.507993 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 10:17:08.508120 (+ 127us) server_negotiation.cc:407] Connection header received
0619 10:17:08.508160 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:17:08.508164 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:17:08.508178 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:17:08.508242 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 10:17:08.508429 (+ 187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:08.509314 (+ 885us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:08.510855 (+ 1541us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:08.511529 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:08.511588 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:08.511648 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0619 10:17:08.511929 (+ 281us) server_negotiation.cc:299] Negotiation successful
0619 10:17:08.511978 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":144,"thread_start_us":103,"threads_started":1}
I20260619 10:17:08.511893 1863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:08.507722 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.65:42851 (local address 127.0.0.1:52288)
0619 10:17:08.507993 (+ 271us) negotiation.cc:107] Waiting for socket to connect
0619 10:17:08.508003 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 10:17:08.508063 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 10:17:08.508313 (+ 250us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 10:17:08.508313 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 10:17:08.508315 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 10:17:08.508363 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:08.508364 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:08.509419 (+ 1055us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:08.509420 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:08.510786 (+ 1366us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:08.510788 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:08.511679 (+ 891us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:08.511680 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:08.511731 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:08.511734 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 10:17:08.511780 (+ 46us) client_negotiation.cc:240] Negotiation successful
0619 10:17:08.511818 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":257,"mutex_wait_us":30,"thread_start_us":46,"threads_started":1}
I20260619 10:17:08.512404 1864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:08.507827 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51468 (local address 127.0.206.67:37739)
0619 10:17:08.508059 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0619 10:17:08.508063 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 10:17:08.508073 (+ 10us) server_negotiation.cc:407] Connection header received
0619 10:17:08.508121 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:17:08.508122 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:17:08.508140 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:17:08.508219 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 10:17:08.508365 (+ 146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:08.509430 (+ 1065us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:08.510932 (+ 1502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:08.511410 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:08.511510 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:08.511565 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0619 10:17:08.511848 (+ 283us) server_negotiation.cc:299] Negotiation successful
0619 10:17:08.511896 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":205,"thread_start_us":63,"threads_started":1}
I20260619 10:17:08.512737 1654 tablet_service.cc:1467] Processing CreateTablet for tablet 892953c8ba304d168b1b3b1ebac1185a (DEFAULT_TABLE table=test-table [id=e51549f8fefe4a1baf876d26729f8e18]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 10:17:08.512756 1520 tablet_service.cc:1467] Processing CreateTablet for tablet 892953c8ba304d168b1b3b1ebac1185a (DEFAULT_TABLE table=test-table [id=e51549f8fefe4a1baf876d26729f8e18]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 10:17:08.512931 1788 tablet_service.cc:1467] Processing CreateTablet for tablet 892953c8ba304d168b1b3b1ebac1185a (DEFAULT_TABLE table=test-table [id=e51549f8fefe4a1baf876d26729f8e18]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 10:17:08.513083 1654 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 892953c8ba304d168b1b3b1ebac1185a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 10:17:08.513091 1520 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 892953c8ba304d168b1b3b1ebac1185a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 10:17:08.513207 1788 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 892953c8ba304d168b1b3b1ebac1185a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 10:17:08.515913 1868 tablet_bootstrap.cc:492] T 892953c8ba304d168b1b3b1ebac1185a P ef84a4869075487cba970c5831c8a912: Bootstrap starting.
I20260619 10:17:08.516324 1869 tablet_bootstrap.cc:492] T 892953c8ba304d168b1b3b1ebac1185a P c3691603b6c044f8aca1a41fec1bc69e: Bootstrap starting.
I20260619 10:17:08.516970 1868 tablet_bootstrap.cc:654] T 892953c8ba304d168b1b3b1ebac1185a P ef84a4869075487cba970c5831c8a912: Neither blocks nor log segments found. Creating new log.
I20260619 10:17:08.517383 1869 tablet_bootstrap.cc:654] T 892953c8ba304d168b1b3b1ebac1185a P c3691603b6c044f8aca1a41fec1bc69e: Neither blocks nor log segments found. Creating new log.
I20260619 10:17:08.517554 1868 log.cc:824] T 892953c8ba304d168b1b3b1ebac1185a P ef84a4869075487cba970c5831c8a912: Log is configured to *not* fsync() on all Append() calls
I20260619 10:17:08.517920 1869 log.cc:824] T 892953c8ba304d168b1b3b1ebac1185a P c3691603b6c044f8aca1a41fec1bc69e: Log is configured to *not* fsync() on all Append() calls
I20260619 10:17:08.518218 1870 tablet_bootstrap.cc:492] T 892953c8ba304d168b1b3b1ebac1185a P eb2a6db87881499c8c5301b38ae8d850: Bootstrap starting.
I20260619 10:17:08.518345 1868 tablet_bootstrap.cc:492] T 892953c8ba304d168b1b3b1ebac1185a P ef84a4869075487cba970c5831c8a912: No bootstrap required, opened a new log
I20260619 10:17:08.518424 1868 ts_tablet_manager.cc:1397] T 892953c8ba304d168b1b3b1ebac1185a P ef84a4869075487cba970c5831c8a912: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260619 10:17:08.518708 1869 tablet_bootstrap.cc:492] T 892953c8ba304d168b1b3b1ebac1185a P c3691603b6c044f8aca1a41fec1bc69e: No bootstrap required, opened a new log
I20260619 10:17:08.518777 1869 ts_tablet_manager.cc:1397] T 892953c8ba304d168b1b3b1ebac1185a P c3691603b6c044f8aca1a41fec1bc69e: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260619 10:17:08.518817 1870 tablet_bootstrap.cc:654] T 892953c8ba304d168b1b3b1ebac1185a P eb2a6db87881499c8c5301b38ae8d850: Neither blocks nor log segments found. Creating new log.
I20260619 10:17:08.519307 1870 log.cc:824] T 892953c8ba304d168b1b3b1ebac1185a P eb2a6db87881499c8c5301b38ae8d850: Log is configured to *not* fsync() on all Append() calls
I20260619 10:17:08.520089 1868 raft_consensus.cc:348] T 892953c8ba304d168b1b3b1ebac1185a P ef84a4869075487cba970c5831c8a912 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb2a6db87881499c8c5301b38ae8d850" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 37739 } } peers { permanent_uuid: "c3691603b6c044f8aca1a41fec1bc69e" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 42881 } } peers { permanent_uuid: "ef84a4869075487cba970c5831c8a912" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 42851 } }
I20260619 10:17:08.520268 1868 raft_consensus.cc:374] T 892953c8ba304d168b1b3b1ebac1185a P ef84a4869075487cba970c5831c8a912 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 10:17:08.520320 1868 raft_consensus.cc:729] T 892953c8ba304d168b1b3b1ebac1185a P ef84a4869075487cba970c5831c8a912 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ef84a4869075487cba970c5831c8a912, State: Initialized, Role: FOLLOWER
I20260619 10:17:08.520429 1868 consensus_queue.cc:260] T 892953c8ba304d168b1b3b1ebac1185a P ef84a4869075487cba970c5831c8a912 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb2a6db87881499c8c5301b38ae8d850" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 37739 } } peers { permanent_uuid: "c3691603b6c044f8aca1a41fec1bc69e" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 42881 } } peers { permanent_uuid: "ef84a4869075487cba970c5831c8a912" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 42851 } }
I20260619 10:17:08.520550 1869 raft_consensus.cc:348] T 892953c8ba304d168b1b3b1ebac1185a P c3691603b6c044f8aca1a41fec1bc69e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb2a6db87881499c8c5301b38ae8d850" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 37739 } } peers { permanent_uuid: "c3691603b6c044f8aca1a41fec1bc69e" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 42881 } } peers { permanent_uuid: "ef84a4869075487cba970c5831c8a912" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 42851 } }
I20260619 10:17:08.520746 1869 raft_consensus.cc:374] T 892953c8ba304d168b1b3b1ebac1185a P c3691603b6c044f8aca1a41fec1bc69e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 10:17:08.520737 1586 heartbeater.cc:503] Master 127.0.206.126:36805 was elected leader, sending a full tablet report...
I20260619 10:17:08.520735 1868 ts_tablet_manager.cc:1428] T 892953c8ba304d168b1b3b1ebac1185a P ef84a4869075487cba970c5831c8a912: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260619 10:17:08.520790 1869 raft_consensus.cc:729] T 892953c8ba304d168b1b3b1ebac1185a P c3691603b6c044f8aca1a41fec1bc69e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c3691603b6c044f8aca1a41fec1bc69e, State: Initialized, Role: FOLLOWER
I20260619 10:17:08.520898 1869 consensus_queue.cc:260] T 892953c8ba304d168b1b3b1ebac1185a P c3691603b6c044f8aca1a41fec1bc69e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb2a6db87881499c8c5301b38ae8d850" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 37739 } } peers { permanent_uuid: "c3691603b6c044f8aca1a41fec1bc69e" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 42881 } } peers { permanent_uuid: "ef84a4869075487cba970c5831c8a912" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 42851 } }
I20260619 10:17:08.521145 1869 ts_tablet_manager.cc:1428] T 892953c8ba304d168b1b3b1ebac1185a P c3691603b6c044f8aca1a41fec1bc69e: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260619 10:17:08.521152 1720 heartbeater.cc:503] Master 127.0.206.126:36805 was elected leader, sending a full tablet report...
I20260619 10:17:08.521334 1870 tablet_bootstrap.cc:492] T 892953c8ba304d168b1b3b1ebac1185a P eb2a6db87881499c8c5301b38ae8d850: No bootstrap required, opened a new log
I20260619 10:17:08.521391 1870 ts_tablet_manager.cc:1397] T 892953c8ba304d168b1b3b1ebac1185a P eb2a6db87881499c8c5301b38ae8d850: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260619 10:17:08.522953 1870 raft_consensus.cc:348] T 892953c8ba304d168b1b3b1ebac1185a P eb2a6db87881499c8c5301b38ae8d850 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb2a6db87881499c8c5301b38ae8d850" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 37739 } } peers { permanent_uuid: "c3691603b6c044f8aca1a41fec1bc69e" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 42881 } } peers { permanent_uuid: "ef84a4869075487cba970c5831c8a912" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 42851 } }
I20260619 10:17:08.523070 1870 raft_consensus.cc:374] T 892953c8ba304d168b1b3b1ebac1185a P eb2a6db87881499c8c5301b38ae8d850 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 10:17:08.523090 1870 raft_consensus.cc:729] T 892953c8ba304d168b1b3b1ebac1185a P eb2a6db87881499c8c5301b38ae8d850 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb2a6db87881499c8c5301b38ae8d850, State: Initialized, Role: FOLLOWER
I20260619 10:17:08.523213 1870 consensus_queue.cc:260] T 892953c8ba304d168b1b3b1ebac1185a P eb2a6db87881499c8c5301b38ae8d850 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb2a6db87881499c8c5301b38ae8d850" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 37739 } } peers { permanent_uuid: "c3691603b6c044f8aca1a41fec1bc69e" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 42881 } } peers { permanent_uuid: "ef84a4869075487cba970c5831c8a912" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 42851 } }
I20260619 10:17:08.523487 1870 ts_tablet_manager.cc:1428] T 892953c8ba304d168b1b3b1ebac1185a P eb2a6db87881499c8c5301b38ae8d850: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260619 10:17:08.523571 1854 heartbeater.cc:503] Master 127.0.206.126:36805 was elected leader, sending a full tablet report...
W20260619 10:17:08.567773 1721 tablet.cc:2369] T 892953c8ba304d168b1b3b1ebac1185a P c3691603b6c044f8aca1a41fec1bc69e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 10:17:08.606825 1876 raft_consensus.cc:482] T 892953c8ba304d168b1b3b1ebac1185a P eb2a6db87881499c8c5301b38ae8d850 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 10:17:08.606981 1876 raft_consensus.cc:504] T 892953c8ba304d168b1b3b1ebac1185a P eb2a6db87881499c8c5301b38ae8d850 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb2a6db87881499c8c5301b38ae8d850" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 37739 } } peers { permanent_uuid: "c3691603b6c044f8aca1a41fec1bc69e" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 42881 } } peers { permanent_uuid: "ef84a4869075487cba970c5831c8a912" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 42851 } }
I20260619 10:17:08.607455 1876 leader_election.cc:288] T 892953c8ba304d168b1b3b1ebac1185a P eb2a6db87881499c8c5301b38ae8d850 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c3691603b6c044f8aca1a41fec1bc69e (127.0.206.66:42881), ef84a4869075487cba970c5831c8a912 (127.0.206.65:42851)
I20260619 10:17:08.611248 1856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:08.607605 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.66:42881 (local address 127.0.206.67:44373)
0619 10:17:08.607676 (+ 71us) negotiation.cc:107] Waiting for socket to connect
0619 10:17:08.607690 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0619 10:17:08.607765 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 10:17:08.607974 (+ 209us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 10:17:08.607976 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 10:17:08.607977 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 10:17:08.608054 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:08.608056 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:08.608995 (+ 939us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:08.608996 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:08.610100 (+ 1104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:08.610103 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:08.610995 (+ 892us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:08.610996 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:08.611071 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:08.611074 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 10:17:08.611137 (+ 63us) client_negotiation.cc:240] Negotiation successful
0619 10:17:08.611177 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":47}
I20260619 10:17:08.611343 1867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:08.607750 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.67:44373 (local address 127.0.206.66:42881)
0619 10:17:08.607808 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0619 10:17:08.607810 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 10:17:08.607820 (+ 10us) server_negotiation.cc:407] Connection header received
0619 10:17:08.607847 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:17:08.607848 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:17:08.607859 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:17:08.607919 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 10:17:08.608130 (+ 211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:08.608917 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:08.610213 (+ 1296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:08.610886 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:08.610955 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:08.610966 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0619 10:17:08.611165 (+ 199us) server_negotiation.cc:299] Negotiation successful
0619 10:17:08.611244 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260619 10:17:08.611737 1674 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "892953c8ba304d168b1b3b1ebac1185a" candidate_uuid: "eb2a6db87881499c8c5301b38ae8d850" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c3691603b6c044f8aca1a41fec1bc69e" is_pre_election: true
I20260619 10:17:08.611889 1674 raft_consensus.cc:2454] T 892953c8ba304d168b1b3b1ebac1185a P c3691603b6c044f8aca1a41fec1bc69e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb2a6db87881499c8c5301b38ae8d850 in term 0.
I20260619 10:17:08.611931 1866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:08.607712 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.67:56983 (local address 127.0.206.65:42851)
0619 10:17:08.607756 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0619 10:17:08.607758 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 10:17:08.607985 (+ 227us) server_negotiation.cc:407] Connection header received
0619 10:17:08.608043 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:17:08.608044 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:17:08.608055 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:17:08.608133 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 10:17:08.608290 (+ 157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:08.609033 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:08.610855 (+ 1822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:08.611469 (+ 614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:08.611542 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:08.611553 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0619 10:17:08.611821 (+ 268us) server_negotiation.cc:299] Negotiation successful
0619 10:17:08.611865 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260619 10:17:08.611979 1877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:08.607624 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.65:42851 (local address 127.0.206.67:56983)
0619 10:17:08.607936 (+ 312us) negotiation.cc:107] Waiting for socket to connect
0619 10:17:08.607944 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0619 10:17:08.607991 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 10:17:08.608136 (+ 145us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 10:17:08.608137 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 10:17:08.608139 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 10:17:08.608214 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:08.608215 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:08.609128 (+ 913us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:08.609129 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:08.610747 (+ 1618us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:08.610750 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:08.611541 (+ 791us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:08.611542 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:08.611645 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:08.611677 (+ 32us) client_negotiation.cc:769] Sending connection context
0619 10:17:08.611807 (+ 130us) client_negotiation.cc:240] Negotiation successful
0619 10:17:08.611878 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":297,"mutex_wait_us":144,"thread_start_us":113,"threads_started":1}
I20260619 10:17:08.612133 1744 leader_election.cc:302] T 892953c8ba304d168b1b3b1ebac1185a P eb2a6db87881499c8c5301b38ae8d850 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c3691603b6c044f8aca1a41fec1bc69e, eb2a6db87881499c8c5301b38ae8d850; no voters:
I20260619 10:17:08.612246 1876 raft_consensus.cc:2781] T 892953c8ba304d168b1b3b1ebac1185a P eb2a6db87881499c8c5301b38ae8d850 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 10:17:08.612329 1876 raft_consensus.cc:482] T 892953c8ba304d168b1b3b1ebac1185a P eb2a6db87881499c8c5301b38ae8d850 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 10:17:08.612272 1540 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "892953c8ba304d168b1b3b1ebac1185a" candidate_uuid: "eb2a6db87881499c8c5301b38ae8d850" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ef84a4869075487cba970c5831c8a912" is_pre_election: true
I20260619 10:17:08.612351 1876 raft_consensus.cc:3037] T 892953c8ba304d168b1b3b1ebac1185a P eb2a6db87881499c8c5301b38ae8d850 [term 0 FOLLOWER]: Advancing to term 1
I20260619 10:17:08.612423 1540 raft_consensus.cc:2454] T 892953c8ba304d168b1b3b1ebac1185a P ef84a4869075487cba970c5831c8a912 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb2a6db87881499c8c5301b38ae8d850 in term 0.
I20260619 10:17:08.613101 1876 raft_consensus.cc:504] T 892953c8ba304d168b1b3b1ebac1185a P eb2a6db87881499c8c5301b38ae8d850 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb2a6db87881499c8c5301b38ae8d850" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 37739 } } peers { permanent_uuid: "c3691603b6c044f8aca1a41fec1bc69e" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 42881 } } peers { permanent_uuid: "ef84a4869075487cba970c5831c8a912" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 42851 } }
I20260619 10:17:08.613238 1876 leader_election.cc:288] T 892953c8ba304d168b1b3b1ebac1185a P eb2a6db87881499c8c5301b38ae8d850 [CANDIDATE]: Term 1 election: Requested vote from peers c3691603b6c044f8aca1a41fec1bc69e (127.0.206.66:42881), ef84a4869075487cba970c5831c8a912 (127.0.206.65:42851)
I20260619 10:17:08.613380 1674 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "892953c8ba304d168b1b3b1ebac1185a" candidate_uuid: "eb2a6db87881499c8c5301b38ae8d850" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c3691603b6c044f8aca1a41fec1bc69e"
I20260619 10:17:08.613422 1540 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "892953c8ba304d168b1b3b1ebac1185a" candidate_uuid: "eb2a6db87881499c8c5301b38ae8d850" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ef84a4869075487cba970c5831c8a912"
I20260619 10:17:08.613463 1674 raft_consensus.cc:3037] T 892953c8ba304d168b1b3b1ebac1185a P c3691603b6c044f8aca1a41fec1bc69e [term 0 FOLLOWER]: Advancing to term 1
I20260619 10:17:08.613497 1540 raft_consensus.cc:3037] T 892953c8ba304d168b1b3b1ebac1185a P ef84a4869075487cba970c5831c8a912 [term 0 FOLLOWER]: Advancing to term 1
I20260619 10:17:08.614159 1540 raft_consensus.cc:2454] T 892953c8ba304d168b1b3b1ebac1185a P ef84a4869075487cba970c5831c8a912 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb2a6db87881499c8c5301b38ae8d850 in term 1.
I20260619 10:17:08.614166 1674 raft_consensus.cc:2454] T 892953c8ba304d168b1b3b1ebac1185a P c3691603b6c044f8aca1a41fec1bc69e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb2a6db87881499c8c5301b38ae8d850 in term 1.
I20260619 10:17:08.614333 1742 leader_election.cc:302] T 892953c8ba304d168b1b3b1ebac1185a P eb2a6db87881499c8c5301b38ae8d850 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: eb2a6db87881499c8c5301b38ae8d850, ef84a4869075487cba970c5831c8a912; no voters:
I20260619 10:17:08.614424 1876 raft_consensus.cc:2781] T 892953c8ba304d168b1b3b1ebac1185a P eb2a6db87881499c8c5301b38ae8d850 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 10:17:08.614559 1876 raft_consensus.cc:686] T 892953c8ba304d168b1b3b1ebac1185a P eb2a6db87881499c8c5301b38ae8d850 [term 1 LEADER]: Becoming Leader. State: Replica: eb2a6db87881499c8c5301b38ae8d850, State: Running, Role: LEADER
I20260619 10:17:08.614715 1876 consensus_queue.cc:237] T 892953c8ba304d168b1b3b1ebac1185a P eb2a6db87881499c8c5301b38ae8d850 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb2a6db87881499c8c5301b38ae8d850" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 37739 } } peers { permanent_uuid: "c3691603b6c044f8aca1a41fec1bc69e" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 42881 } } peers { permanent_uuid: "ef84a4869075487cba970c5831c8a912" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 42851 } }
I20260619 10:17:08.615505 1395 catalog_manager.cc:5495] T 892953c8ba304d168b1b3b1ebac1185a P eb2a6db87881499c8c5301b38ae8d850 reported cstate change: term changed from 0 to 1, leader changed from <none> to eb2a6db87881499c8c5301b38ae8d850 (127.0.206.67). New cstate: current_term: 1 leader_uuid: "eb2a6db87881499c8c5301b38ae8d850" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb2a6db87881499c8c5301b38ae8d850" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 37739 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c3691603b6c044f8aca1a41fec1bc69e" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 42881 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ef84a4869075487cba970c5831c8a912" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 42851 } health_report { overall_health: UNKNOWN } } }
Jun 19 10:17:08 dist-test-slave-6pt2 krb5kdc[1350](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864227, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.206.65@KRBTEST.COM
I20260619 10:17:08.629652 1866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:08.622998 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52292 (local address 127.0.206.65:42851)
0619 10:17:08.623067 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0619 10:17:08.623068 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 10:17:08.623160 (+ 92us) server_negotiation.cc:407] Connection header received
0619 10:17:08.623208 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:17:08.623208 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:17:08.623217 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:17:08.623253 (+ 36us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:17:08.623724 (+ 471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:08.624146 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:08.624773 (+ 627us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:08.624939 (+ 166us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:08.624985 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:08.627411 (+ 2426us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 10:17:08.627417 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 10:17:08.627417 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 10:17:08.627423 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 10:17:08.628752 (+ 1329us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:17:08.629046 (+ 294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:17:08.629048 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:17:08.629048 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:17:08.629096 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:17:08.629251 (+ 155us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:17:08.629252 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:17:08.629252 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:17:08.629361 (+ 109us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 10:17:08.629455 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0619 10:17:08.629510 (+ 55us) server_negotiation.cc:299] Negotiation successful
0619 10:17:08.629574 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
Jun 19 10:17:08 dist-test-slave-6pt2 krb5kdc[1350](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864228, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Jun 19 10:17:08 dist-test-slave-6pt2 krb5kdc[1350](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864228, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.0.206.65@KRBTEST.COM
I20260619 10:17:08.651062 1866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:08.644533 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52294 (local address 127.0.206.65:42851)
0619 10:17:08.644593 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0619 10:17:08.644595 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 10:17:08.644642 (+ 47us) server_negotiation.cc:407] Connection header received
0619 10:17:08.644685 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:17:08.644686 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:17:08.644694 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:17:08.644742 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:17:08.645234 (+ 492us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:08.645735 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:08.646415 (+ 680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:08.646682 (+ 267us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:08.646734 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:08.648927 (+ 2193us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 10:17:08.648929 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 10:17:08.648929 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 10:17:08.648936 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 10:17:08.650055 (+ 1119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:17:08.650348 (+ 293us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:17:08.650349 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:17:08.650349 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:17:08.650393 (+ 44us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:17:08.650560 (+ 167us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:17:08.650560 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:17:08.650561 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:17:08.650712 (+ 151us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 10:17:08.650784 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0619 10:17:08.650924 (+ 140us) server_negotiation.cc:299] Negotiation successful
0619 10:17:08.650980 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
Jun 19 10:17:08 dist-test-slave-6pt2 krb5kdc[1350](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864228, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Jun 19 10:17:08 dist-test-slave-6pt2 krb5kdc[1350](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864228, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.206.65@KRBTEST.COM
I20260619 10:17:08.671291 1866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:08.665076 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52304 (local address 127.0.206.65:42851)
0619 10:17:08.665160 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0619 10:17:08.665162 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 10:17:08.665182 (+ 20us) server_negotiation.cc:407] Connection header received
0619 10:17:08.665234 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:17:08.665235 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:17:08.665243 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:17:08.665295 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:17:08.665753 (+ 458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:08.666196 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:08.666920 (+ 724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:08.667084 (+ 164us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:08.667132 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:08.669266 (+ 2134us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 10:17:08.669267 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 10:17:08.669267 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 10:17:08.669273 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 10:17:08.670387 (+ 1114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:17:08.670762 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:17:08.670762 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:17:08.670763 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:17:08.670810 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:17:08.670953 (+ 143us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:17:08.670953 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:17:08.670953 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:17:08.671029 (+ 76us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 10:17:08.671093 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0619 10:17:08.671176 (+ 83us) server_negotiation.cc:299] Negotiation successful
0619 10:17:08.671227 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
W20260619 10:17:08.671779 1500 tablet_service.cc:2908] Rejecting scan request for tablet 892953c8ba304d168b1b3b1ebac1185a: Uninitialized: clean time has not yet been initialized
I20260619 10:17:08.672513 825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTU9OP6/build/release/bin/kudu with pid 1457
I20260619 10:17:08.677400 825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTU9OP6/build/release/bin/kudu with pid 1591
I20260619 10:17:08.681442 825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTU9OP6/build/release/bin/kudu with pid 1725
I20260619 10:17:08.685703 825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTU9OP6/build/release/bin/kudu with pid 1363
2026-06-19T10:17:08Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (1964 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781864224577058-825-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 10:17:08 dist-test-slave-6pt2 krb5kdc[1892](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 10:17:08 dist-test-slave-6pt2 krb5kdc[1892](info): set up 2 sockets
Jun 19 10:17:08 dist-test-slave-6pt2 krb5kdc[1892](info): commencing operation
krb5kdc: starting...
W20260619 10:17:09.698118 825 mini_kdc.cc:121] Time spent starting KDC: real 0.992s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781864224577058-825-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781864224577058-825-0/krb5kdc/test-user.keytab.
Jun 19 10:17:09 dist-test-slave-6pt2 krb5kdc[1892](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864229, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T10:17:09Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T10:17:09Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.206.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.206.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.206.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781864224577058-825-0/krb5kdc/kudu_127.0.206.126.keytab.
Entry for principal kudu/127.0.206.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781864224577058-825-0/krb5kdc/kudu_127.0.206.126.keytab.
I20260619 10:17:09.807410 825 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTU9OP6/build/release/bin/kudu
/tmp/dist-test-taskTU9OP6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781864224577058-825-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781864224577058-825-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781864224577058-825-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781864224577058-825-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.206.126:38679
--webserver_interface=127.0.206.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781864224577058-825-0/krb5kdc/kudu_127.0.206.126.keytab
--principal=kudu/127.0.206.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.206.84:45993
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.206.126:38679
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781864224577058-825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781864224577058-825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781864224577058-825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 10:17:09.884874 1905 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 10:17:09.885102 1905 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 10:17:09.885136 1905 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 10:17:09.886237 1905 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 10:17:09.886312 1905 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260619 10:17:09.886332 1905 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 10:17:09.886376 1905 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 10:17:09.886406 1905 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 10:17:09.886430 1905 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 10:17:09.887980 1905 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.206.84:45993
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781864224577058-825-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781864224577058-825-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.206.126:38679
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781864224577058-825-0/krb5kdc/kudu_127.0.206.126.keytab
--principal=kudu/127.0.206.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.206.126:38679
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781864224577058-825-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.206.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1905
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781864224577058-825-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 09:32:44 UTC on ccycloud.quasar-gbtldp.root.comops.site
build id 4455440
I20260619 10:17:09.888240 1905 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 10:17:09.888463 1905 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 10:17:09.893229 1910 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 10:17:09.893253 1913 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 10:17:09.893270 1911 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 10:17:09.893282 1905 server_base.cc:1034] running on GCE node
I20260619 10:17:09.893628 1905 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 10:17:09.893908 1905 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 10:17:09.895068 1905 hybrid_clock.cc:648] HybridClock initialized: now 1781864229895055 us; error 44 us; skew 500 ppm
Jun 19 10:17:09 dist-test-slave-6pt2 krb5kdc[1892](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864229, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 10:17:09.900672 1905 init.cc:377] Logged in from keytab as kudu/127.0.206.126@KRBTEST.COM (short username kudu)
I20260619 10:17:09.901135 1905 webserver.cc:460] Webserver started at http://127.0.206.126:35129/ using document root <none> and password file <none>
I20260619 10:17:09.901371 1905 fs_manager.cc:359] Metadata directory not provided
I20260619 10:17:09.901438 1905 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 10:17:09.901597 1905 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 10:17:09.902721 1905 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781864224577058-825-0/minicluster-data/master-0/data/instance:
uuid: "1f1f90ebcf834d238665f58d7af3e329"
format_stamp: "Formatted at 2026-06-19 10:17:09 on dist-test-slave-6pt2"
server_key: "c2695e6507505582aed8156639dfdfb9"
server_key_iv: "ae36cf174613120699c8095665f096fb"
server_key_version: "encryptionkey@0"
I20260619 10:17:09.903077 1905 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781864224577058-825-0/minicluster-data/master-0/wal/instance:
uuid: "1f1f90ebcf834d238665f58d7af3e329"
format_stamp: "Formatted at 2026-06-19 10:17:09 on dist-test-slave-6pt2"
server_key: "c2695e6507505582aed8156639dfdfb9"
server_key_iv: "ae36cf174613120699c8095665f096fb"
server_key_version: "encryptionkey@0"
I20260619 10:17:09.904341 1905 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 10:17:09.905056 1920 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 10:17:09.905212 1905 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 10:17:09.905306 1905 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781864224577058-825-0/minicluster-data/master-0/data,/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781864224577058-825-0/minicluster-data/master-0/wal
uuid: "1f1f90ebcf834d238665f58d7af3e329"
format_stamp: "Formatted at 2026-06-19 10:17:09 on dist-test-slave-6pt2"
server_key: "c2695e6507505582aed8156639dfdfb9"
server_key_iv: "ae36cf174613120699c8095665f096fb"
server_key_version: "encryptionkey@0"
I20260619 10:17:09.905380 1905 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781864224577058-825-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781864224577058-825-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781864224577058-825-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 10:17:09.915496 1905 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 10:17:09.917750 1905 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 10:17:09.917910 1905 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 10:17:09.922640 1905 rpc_server.cc:307] RPC server started. Bound to: 127.0.206.126:38679
I20260619 10:17:09.922712 1982 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.206.126:38679 every 8 connection(s)
I20260619 10:17:09.923051 1905 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781864224577058-825-0/minicluster-data/master-0/data/info.pb
I20260619 10:17:09.923686 1983 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 10:17:09.925894 1983 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1f1f90ebcf834d238665f58d7af3e329: Bootstrap starting.
I20260619 10:17:09.926493 1983 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1f1f90ebcf834d238665f58d7af3e329: Neither blocks nor log segments found. Creating new log.
I20260619 10:17:09.926807 1983 log.cc:824] T 00000000000000000000000000000000 P 1f1f90ebcf834d238665f58d7af3e329: Log is configured to *not* fsync() on all Append() calls
I20260619 10:17:09.927484 1983 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1f1f90ebcf834d238665f58d7af3e329: No bootstrap required, opened a new log
I20260619 10:17:09.929454 1983 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1f1f90ebcf834d238665f58d7af3e329 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1f1f90ebcf834d238665f58d7af3e329" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 38679 } }
I20260619 10:17:09.929581 1983 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1f1f90ebcf834d238665f58d7af3e329 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 10:17:09.929603 1983 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1f1f90ebcf834d238665f58d7af3e329 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f1f90ebcf834d238665f58d7af3e329, State: Initialized, Role: FOLLOWER
I20260619 10:17:09.929768 1983 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1f1f90ebcf834d238665f58d7af3e329 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1f1f90ebcf834d238665f58d7af3e329" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 38679 } }
I20260619 10:17:09.929827 1983 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1f1f90ebcf834d238665f58d7af3e329 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 10:17:09.929847 1983 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1f1f90ebcf834d238665f58d7af3e329 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 10:17:09.929920 1983 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1f1f90ebcf834d238665f58d7af3e329 [term 0 FOLLOWER]: Advancing to term 1
I20260619 10:17:09.930408 1983 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1f1f90ebcf834d238665f58d7af3e329 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1f1f90ebcf834d238665f58d7af3e329" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 38679 } }
I20260619 10:17:09.930510 1983 leader_election.cc:302] T 00000000000000000000000000000000 P 1f1f90ebcf834d238665f58d7af3e329 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1f1f90ebcf834d238665f58d7af3e329; no voters:
I20260619 10:17:09.930734 1983 leader_election.cc:288] T 00000000000000000000000000000000 P 1f1f90ebcf834d238665f58d7af3e329 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 10:17:09.930807 1986 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1f1f90ebcf834d238665f58d7af3e329 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 10:17:09.930948 1986 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1f1f90ebcf834d238665f58d7af3e329 [term 1 LEADER]: Becoming Leader. State: Replica: 1f1f90ebcf834d238665f58d7af3e329, State: Running, Role: LEADER
I20260619 10:17:09.930990 1983 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1f1f90ebcf834d238665f58d7af3e329 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 10:17:09.931044 1986 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1f1f90ebcf834d238665f58d7af3e329 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1f1f90ebcf834d238665f58d7af3e329" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 38679 } }
I20260619 10:17:09.931404 1987 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1f1f90ebcf834d238665f58d7af3e329 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1f1f90ebcf834d238665f58d7af3e329" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1f1f90ebcf834d238665f58d7af3e329" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 38679 } } }
I20260619 10:17:09.931578 1987 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1f1f90ebcf834d238665f58d7af3e329 [sys.catalog]: This master's current role is: LEADER
I20260619 10:17:09.931823 825 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTU9OP6/build/release/bin/kudu as pid 1905
I20260619 10:17:09.931418 1988 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1f1f90ebcf834d238665f58d7af3e329 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1f1f90ebcf834d238665f58d7af3e329. Latest consensus state: current_term: 1 leader_uuid: "1f1f90ebcf834d238665f58d7af3e329" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1f1f90ebcf834d238665f58d7af3e329" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 38679 } } }
I20260619 10:17:09.931911 825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781864224577058-825-0/minicluster-data/master-0/wal/instance
I20260619 10:17:09.931986 1988 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1f1f90ebcf834d238665f58d7af3e329 [sys.catalog]: This master's current role is: LEADER
I20260619 10:17:09.932093 825 external_mini_cluster.cc:1428] Setting key e843744f2d7a7fa884f23f4c13f5f593
I20260619 10:17:09.932260 1995 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 10:17:09.932787 1995 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 10:17:09.935139 1995 catalog_manager.cc:1347] Generated new cluster ID: 381facb665d54ff58cd7d7bb9deeecf9
I20260619 10:17:09.935194 1995 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Jun 19 10:17:09 dist-test-slave-6pt2 krb5kdc[1892](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864229, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.206.126@KRBTEST.COM
I20260619 10:17:09.942322 1995 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 10:17:09.942885 1995 catalog_manager.cc:1504] Loading token signing keys...
I20260619 10:17:09.942967 2005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:09.932572 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38510 (local address 127.0.206.126:38679)
0619 10:17:09.932712 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0619 10:17:09.932722 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0619 10:17:09.932755 (+ 33us) server_negotiation.cc:407] Connection header received
0619 10:17:09.932879 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:17:09.932884 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:17:09.932925 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:17:09.933061 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:17:09.933796 (+ 735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:09.934760 (+ 964us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:09.935809 (+ 1049us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:09.936444 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:09.936558 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:09.939258 (+ 2700us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 10:17:09.939262 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 10:17:09.939262 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 10:17:09.939274 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0619 10:17:09.941193 (+ 1919us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:17:09.941582 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:17:09.941585 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:17:09.941585 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:17:09.941687 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:17:09.941902 (+ 215us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:17:09.941903 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:17:09.941904 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:17:09.942184 (+ 280us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 10:17:09.942340 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0619 10:17:09.942619 (+ 279us) server_negotiation.cc:299] Negotiation successful
0619 10:17:09.942758 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":96,"thread_start_us":69,"threads_started":1}
I20260619 10:17:09.948962 1995 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1f1f90ebcf834d238665f58d7af3e329: Generated new TSK 0
I20260619 10:17:09.949136 1995 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.0.206.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.206.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.206.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781864224577058-825-0/krb5kdc/kudu_127.0.206.65.keytab.
Entry for principal kudu/127.0.206.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781864224577058-825-0/krb5kdc/kudu_127.0.206.65.keytab.
I20260619 10:17:09.972807 825 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTU9OP6/build/release/bin/kudu
/tmp/dist-test-taskTU9OP6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781864224577058-825-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781864224577058-825-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781864224577058-825-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781864224577058-825-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.206.65:0
--local_ip_for_outbound_sockets=127.0.206.65
--webserver_interface=127.0.206.65
--webserver_port=0
--tserver_master_addrs=127.0.206.126:38679
--keytab_file=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781864224577058-825-0/krb5kdc/kudu_127.0.206.65.keytab
--principal=kudu/127.0.206.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.206.84:45993
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781864224577058-825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781864224577058-825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781864224577058-825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 10:17:10.051482 2009 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 10:17:10.051674 2009 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 10:17:10.051697 2009 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 10:17:10.052860 2009 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 10:17:10.052899 2009 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 10:17:10.052920 2009 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260619 10:17:10.052937 2009 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.206.65
I20260619 10:17:10.054390 2009 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.206.84:45993
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781864224577058-825-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781864224577058-825-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781864224577058-825-0/krb5kdc/kudu_127.0.206.65.keytab
--principal=kudu/127.0.206.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.206.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781864224577058-825-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.206.65
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.0.206.126:38679
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2009
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.206.65
--log_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781864224577058-825-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 09:32:44 UTC on ccycloud.quasar-gbtldp.root.comops.site
build id 4455440
I20260619 10:17:10.054646 2009 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 10:17:10.054869 2009 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 10:17:10.059720 2017 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 10:17:10.059767 2009 server_base.cc:1034] running on GCE node
W20260619 10:17:10.059728 2014 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 10:17:10.059738 2015 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 10:17:10.060163 2009 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 10:17:10.060460 2009 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 10:17:10.061615 2009 hybrid_clock.cc:648] HybridClock initialized: now 1781864230061615 us; error 33 us; skew 500 ppm
Jun 19 10:17:10 dist-test-slave-6pt2 krb5kdc[1892](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864230, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 10:17:10.064029 2009 init.cc:377] Logged in from keytab as kudu/127.0.206.65@KRBTEST.COM (short username kudu)
I20260619 10:17:10.064442 2009 webserver.cc:460] Webserver started at http://127.0.206.65:39591/ using document root <none> and password file <none>
I20260619 10:17:10.064651 2009 fs_manager.cc:359] Metadata directory not provided
I20260619 10:17:10.064715 2009 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 10:17:10.064862 2009 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 10:17:10.065842 2009 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781864224577058-825-0/minicluster-data/ts-0/data/instance:
uuid: "a38cfaebaa9a4ae8966419dc8abbd721"
format_stamp: "Formatted at 2026-06-19 10:17:10 on dist-test-slave-6pt2"
server_key: "efa685d1dacc32f1120855c7dbb913b8"
server_key_iv: "dfd9b10b7190440faf228382bcf943d3"
server_key_version: "encryptionkey@0"
I20260619 10:17:10.066192 2009 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781864224577058-825-0/minicluster-data/ts-0/wal/instance:
uuid: "a38cfaebaa9a4ae8966419dc8abbd721"
format_stamp: "Formatted at 2026-06-19 10:17:10 on dist-test-slave-6pt2"
server_key: "efa685d1dacc32f1120855c7dbb913b8"
server_key_iv: "dfd9b10b7190440faf228382bcf943d3"
server_key_version: "encryptionkey@0"
I20260619 10:17:10.067581 2009 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260619 10:17:10.068228 2024 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 10:17:10.068396 2009 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260619 10:17:10.068490 2009 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781864224577058-825-0/minicluster-data/ts-0/data,/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781864224577058-825-0/minicluster-data/ts-0/wal
uuid: "a38cfaebaa9a4ae8966419dc8abbd721"
format_stamp: "Formatted at 2026-06-19 10:17:10 on dist-test-slave-6pt2"
server_key: "efa685d1dacc32f1120855c7dbb913b8"
server_key_iv: "dfd9b10b7190440faf228382bcf943d3"
server_key_version: "encryptionkey@0"
I20260619 10:17:10.068564 2009 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781864224577058-825-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781864224577058-825-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781864224577058-825-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 10:17:10.090672 2009 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 10:17:10.092983 2009 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 10:17:10.093170 2009 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 10:17:10.093676 2009 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 10:17:10.093739 2009 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 10:17:10.093802 2009 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 10:17:10.093833 2009 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Jun 19 10:17:10 dist-test-slave-6pt2 krb5kdc[1892](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864230, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.65@KRBTEST.COM for kudu/127.0.206.126@KRBTEST.COM
I20260619 10:17:10.100623 2009 rpc_server.cc:307] RPC server started. Bound to: 127.0.206.65:34017
I20260619 10:17:10.100692 2139 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.206.65:34017 every 8 connection(s)
I20260619 10:17:10.101001 2009 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781864224577058-825-0/minicluster-data/ts-0/data/info.pb
I20260619 10:17:10.103753 2005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:10.094203 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.65:38803 (local address 127.0.206.126:38679)
0619 10:17:10.094244 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0619 10:17:10.094246 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 10:17:10.094461 (+ 215us) server_negotiation.cc:407] Connection header received
0619 10:17:10.095053 (+ 592us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:17:10.095054 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:17:10.095067 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:17:10.095122 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:17:10.095976 (+ 854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:10.096801 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:10.098370 (+ 1569us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:10.098724 (+ 354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:10.098796 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:10.101006 (+ 2210us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 10:17:10.101008 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 10:17:10.101008 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 10:17:10.101019 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0619 10:17:10.102314 (+ 1295us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:17:10.102790 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:17:10.102791 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:17:10.102791 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:17:10.102855 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:17:10.103106 (+ 251us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:17:10.103107 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:17:10.103107 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:17:10.103223 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 10:17:10.103296 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0619 10:17:10.103573 (+ 277us) server_negotiation.cc:299] Negotiation successful
0619 10:17:10.103662 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260619 10:17:10.103945 2033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:10.094202 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:38679 (local address 127.0.206.65:38803)
0619 10:17:10.094347 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0619 10:17:10.094381 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0619 10:17:10.094945 (+ 564us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 10:17:10.095198 (+ 253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 10:17:10.095202 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 10:17:10.095593 (+ 391us) client_negotiation.cc:189] Negotiated authn=SASL
0619 10:17:10.095887 (+ 294us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:10.095894 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:10.096885 (+ 991us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:10.096886 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:10.098278 (+ 1392us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:10.098280 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:10.098805 (+ 525us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:10.098805 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:10.098935 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:10.099446 (+ 511us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 10:17:10.099456 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0619 10:17:10.100909 (+ 1453us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 10:17:10.102400 (+ 1491us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 10:17:10.102406 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 10:17:10.102409 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 10:17:10.102697 (+ 288us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 10:17:10.102923 (+ 226us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 10:17:10.102923 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 10:17:10.102924 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 10:17:10.103038 (+ 114us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 10:17:10.103299 (+ 261us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 10:17:10.103306 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 10:17:10.103438 (+ 132us) client_negotiation.cc:769] Sending connection context
0619 10:17:10.103552 (+ 114us) client_negotiation.cc:240] Negotiation successful
0619 10:17:10.103701 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":88,"threads_started":1}
I20260619 10:17:10.104434 2140 heartbeater.cc:347] Connected to a master server at 127.0.206.126:38679
I20260619 10:17:10.104516 2140 heartbeater.cc:464] Registering TS with master...
I20260619 10:17:10.104689 2140 heartbeater.cc:511] Master 127.0.206.126:38679 requested a full tablet report, sending...
I20260619 10:17:10.105316 1936 ts_manager.cc:194] Registered new tserver with Master: a38cfaebaa9a4ae8966419dc8abbd721 (127.0.206.65:34017)
I20260619 10:17:10.106021 1936 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.206.65@KRBTEST.COM'} at 127.0.206.65:38803
I20260619 10:17:10.107263 825 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTU9OP6/build/release/bin/kudu as pid 2009
I20260619 10:17:10.107367 825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781864224577058-825-0/minicluster-data/ts-0/wal/instance
I20260619 10:17:10.107537 825 external_mini_cluster.cc:1428] Setting key c58caffbf0e618db38227fedf1933992
I20260619 10:17:10.111325 2005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:10.105603 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.65:42015 (local address 127.0.206.126:38679)
0619 10:17:10.105642 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0619 10:17:10.105644 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 10:17:10.105653 (+ 9us) server_negotiation.cc:407] Connection header received
0619 10:17:10.105675 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:17:10.105676 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:17:10.105685 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:17:10.105739 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:17:10.106042 (+ 303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:10.106801 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:10.107635 (+ 834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:10.107907 (+ 272us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:10.107966 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:10.108845 (+ 879us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 10:17:10.108847 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 10:17:10.108847 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 10:17:10.108854 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 10:17:10.110226 (+ 1372us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:17:10.110656 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:17:10.110657 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:17:10.110657 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:17:10.110716 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:17:10.110916 (+ 200us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:17:10.110916 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:17:10.110917 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:17:10.111007 (+ 90us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 10:17:10.111055 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0619 10:17:10.111209 (+ 154us) server_negotiation.cc:299] Negotiation successful
0619 10:17:10.111268 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260619 10:17:10.111459 2146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:10.105423 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:38679 (local address 127.0.206.65:42015)
0619 10:17:10.105564 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0619 10:17:10.105576 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 10:17:10.105630 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 10:17:10.105744 (+ 114us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 10:17:10.105744 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 10:17:10.105919 (+ 175us) client_negotiation.cc:189] Negotiated authn=SASL
0619 10:17:10.105965 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:10.105967 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:10.106899 (+ 932us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:10.106900 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:10.107534 (+ 634us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:10.107536 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:10.107980 (+ 444us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:10.107981 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:10.108044 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:10.108481 (+ 437us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 10:17:10.108485 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 10:17:10.108737 (+ 252us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 10:17:10.110304 (+ 1567us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 10:17:10.110305 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 10:17:10.110306 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 10:17:10.110551 (+ 245us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 10:17:10.110784 (+ 233us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 10:17:10.110784 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 10:17:10.110785 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 10:17:10.110857 (+ 72us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 10:17:10.111067 (+ 210us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 10:17:10.111068 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 10:17:10.111119 (+ 51us) client_negotiation.cc:769] Sending connection context
0619 10:17:10.111195 (+ 76us) client_negotiation.cc:240] Negotiation successful
0619 10:17:10.111306 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":79,"threads_started":1}
WARNING: no policy specified for kudu/127.0.206.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.206.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.206.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781864224577058-825-0/krb5kdc/kudu_127.0.206.66.keytab.
Entry for principal kudu/127.0.206.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781864224577058-825-0/krb5kdc/kudu_127.0.206.66.keytab.
I20260619 10:17:10.130918 825 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTU9OP6/build/release/bin/kudu
/tmp/dist-test-taskTU9OP6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781864224577058-825-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781864224577058-825-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781864224577058-825-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781864224577058-825-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.206.66:0
--local_ip_for_outbound_sockets=127.0.206.66
--webserver_interface=127.0.206.66
--webserver_port=0
--tserver_master_addrs=127.0.206.126:38679
--keytab_file=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781864224577058-825-0/krb5kdc/kudu_127.0.206.66.keytab
--principal=kudu/127.0.206.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.206.84:45993
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781864224577058-825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781864224577058-825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781864224577058-825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 10:17:10.211043 2149 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 10:17:10.211230 2149 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 10:17:10.211252 2149 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 10:17:10.212327 2149 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 10:17:10.212360 2149 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 10:17:10.212380 2149 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260619 10:17:10.212397 2149 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.206.66
I20260619 10:17:10.213769 2149 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.206.84:45993
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781864224577058-825-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781864224577058-825-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781864224577058-825-0/krb5kdc/kudu_127.0.206.66.keytab
--principal=kudu/127.0.206.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.206.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781864224577058-825-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.206.66
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.0.206.126:38679
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2149
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.206.66
--log_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781864224577058-825-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 09:32:44 UTC on ccycloud.quasar-gbtldp.root.comops.site
build id 4455440
I20260619 10:17:10.213941 2149 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 10:17:10.214097 2149 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 10:17:10.218781 2155 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 10:17:10.218796 2154 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 10:17:10.218967 2157 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 10:17:10.218982 2149 server_base.cc:1034] running on GCE node
I20260619 10:17:10.219239 2149 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 10:17:10.219496 2149 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 10:17:10.220654 2149 hybrid_clock.cc:648] HybridClock initialized: now 1781864230220652 us; error 35 us; skew 500 ppm
Jun 19 10:17:10 dist-test-slave-6pt2 krb5kdc[1892](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864230, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 10:17:10.223027 2149 init.cc:377] Logged in from keytab as kudu/127.0.206.66@KRBTEST.COM (short username kudu)
I20260619 10:17:10.223497 2149 webserver.cc:460] Webserver started at http://127.0.206.66:43521/ using document root <none> and password file <none>
I20260619 10:17:10.223721 2149 fs_manager.cc:359] Metadata directory not provided
I20260619 10:17:10.223786 2149 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 10:17:10.223917 2149 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 10:17:10.224898 2149 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781864224577058-825-0/minicluster-data/ts-1/data/instance:
uuid: "2bfba7e0bfad4fe4a249d625797d1669"
format_stamp: "Formatted at 2026-06-19 10:17:10 on dist-test-slave-6pt2"
server_key: "ff03173b9dd8017b6c97a33e9cff85fc"
server_key_iv: "0146372da378ceb7bfa333277764d64b"
server_key_version: "encryptionkey@0"
I20260619 10:17:10.225255 2149 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781864224577058-825-0/minicluster-data/ts-1/wal/instance:
uuid: "2bfba7e0bfad4fe4a249d625797d1669"
format_stamp: "Formatted at 2026-06-19 10:17:10 on dist-test-slave-6pt2"
server_key: "ff03173b9dd8017b6c97a33e9cff85fc"
server_key_iv: "0146372da378ceb7bfa333277764d64b"
server_key_version: "encryptionkey@0"
I20260619 10:17:10.226473 2149 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260619 10:17:10.227200 2164 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 10:17:10.227362 2149 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260619 10:17:10.227464 2149 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781864224577058-825-0/minicluster-data/ts-1/data,/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781864224577058-825-0/minicluster-data/ts-1/wal
uuid: "2bfba7e0bfad4fe4a249d625797d1669"
format_stamp: "Formatted at 2026-06-19 10:17:10 on dist-test-slave-6pt2"
server_key: "ff03173b9dd8017b6c97a33e9cff85fc"
server_key_iv: "0146372da378ceb7bfa333277764d64b"
server_key_version: "encryptionkey@0"
I20260619 10:17:10.227548 2149 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781864224577058-825-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781864224577058-825-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781864224577058-825-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 10:17:10.239235 2149 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 10:17:10.240870 2149 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 10:17:10.241019 2149 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 10:17:10.241526 2149 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 10:17:10.241593 2149 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 10:17:10.241623 2149 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 10:17:10.241642 2149 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Jun 19 10:17:10 dist-test-slave-6pt2 krb5kdc[1892](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864230, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.66@KRBTEST.COM for kudu/127.0.206.126@KRBTEST.COM
I20260619 10:17:10.248646 2149 rpc_server.cc:307] RPC server started. Bound to: 127.0.206.66:33487
I20260619 10:17:10.248725 2279 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.206.66:33487 every 8 connection(s)
I20260619 10:17:10.248996 2149 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781864224577058-825-0/minicluster-data/ts-1/data/info.pb
I20260619 10:17:10.250540 2005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:10.241869 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.66:52455 (local address 127.0.206.126:38679)
0619 10:17:10.241919 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0619 10:17:10.241921 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 10:17:10.242126 (+ 205us) server_negotiation.cc:407] Connection header received
0619 10:17:10.242628 (+ 502us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:17:10.242630 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:17:10.242642 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:17:10.242695 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:17:10.243257 (+ 562us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:10.243797 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:10.244808 (+ 1011us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:10.245088 (+ 280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:10.245172 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:10.247908 (+ 2736us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 10:17:10.247910 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 10:17:10.247911 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 10:17:10.247921 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0619 10:17:10.249456 (+ 1535us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:17:10.249809 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:17:10.249810 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:17:10.249810 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:17:10.249857 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:17:10.250065 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:17:10.250066 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:17:10.250066 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:17:10.250159 (+ 93us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 10:17:10.250208 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0619 10:17:10.250420 (+ 212us) server_negotiation.cc:299] Negotiation successful
0619 10:17:10.250478 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260619 10:17:10.250705 2173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:10.241843 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:38679 (local address 127.0.206.66:52455)
0619 10:17:10.242013 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0619 10:17:10.242045 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0619 10:17:10.242496 (+ 451us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 10:17:10.242764 (+ 268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 10:17:10.242767 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 10:17:10.242992 (+ 225us) client_negotiation.cc:189] Negotiated authn=SASL
0619 10:17:10.243160 (+ 168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:10.243164 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:10.243887 (+ 723us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:10.243889 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:10.244714 (+ 825us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:10.244716 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:10.245180 (+ 464us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:10.245181 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:10.245312 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:10.245828 (+ 516us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 10:17:10.245841 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0619 10:17:10.247783 (+ 1942us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 10:17:10.249548 (+ 1765us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 10:17:10.249552 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 10:17:10.249553 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 10:17:10.249748 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 10:17:10.249921 (+ 173us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 10:17:10.249922 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 10:17:10.249922 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 10:17:10.250010 (+ 88us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 10:17:10.250213 (+ 203us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 10:17:10.250216 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 10:17:10.250316 (+ 100us) client_negotiation.cc:769] Sending connection context
0619 10:17:10.250396 (+ 80us) client_negotiation.cc:240] Negotiation successful
0619 10:17:10.250503 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":83,"threads_started":1}
I20260619 10:17:10.251179 2280 heartbeater.cc:347] Connected to a master server at 127.0.206.126:38679
I20260619 10:17:10.251260 2280 heartbeater.cc:464] Registering TS with master...
I20260619 10:17:10.251453 2280 heartbeater.cc:511] Master 127.0.206.126:38679 requested a full tablet report, sending...
I20260619 10:17:10.251859 1937 ts_manager.cc:194] Registered new tserver with Master: 2bfba7e0bfad4fe4a249d625797d1669 (127.0.206.66:33487)
I20260619 10:17:10.252326 1937 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.206.66@KRBTEST.COM'} at 127.0.206.66:52455
I20260619 10:17:10.255604 825 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTU9OP6/build/release/bin/kudu as pid 2149
I20260619 10:17:10.255693 825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781864224577058-825-0/minicluster-data/ts-1/wal/instance
I20260619 10:17:10.255867 825 external_mini_cluster.cc:1428] Setting key d5293d11b7f22b5146bd8914b6d5afd6
I20260619 10:17:10.257364 2005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:10.252193 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.66:52913 (local address 127.0.206.126:38679)
0619 10:17:10.252448 (+ 255us) server_negotiation.cc:206] Beginning negotiation
0619 10:17:10.252450 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 10:17:10.252458 (+ 8us) server_negotiation.cc:407] Connection header received
0619 10:17:10.252476 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:17:10.252477 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:17:10.252487 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:17:10.252538 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:17:10.252880 (+ 342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:10.253302 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:10.253928 (+ 626us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:10.254093 (+ 165us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:10.254140 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:10.254970 (+ 830us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 10:17:10.254971 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 10:17:10.254971 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 10:17:10.254976 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0619 10:17:10.256339 (+ 1363us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:17:10.256702 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:17:10.256703 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:17:10.256704 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:17:10.256750 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:17:10.256943 (+ 193us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:17:10.256943 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:17:10.256944 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:17:10.257057 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 10:17:10.257106 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0619 10:17:10.257259 (+ 153us) server_negotiation.cc:299] Negotiation successful
0619 10:17:10.257311 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":238}
I20260619 10:17:10.257480 2286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:10.252130 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:38679 (local address 127.0.206.66:52913)
0619 10:17:10.252241 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0619 10:17:10.252251 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 10:17:10.252303 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 10:17:10.252544 (+ 241us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 10:17:10.252545 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 10:17:10.252743 (+ 198us) client_negotiation.cc:189] Negotiated authn=SASL
0619 10:17:10.252805 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:10.252807 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:10.253386 (+ 579us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:10.253387 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:10.253844 (+ 457us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:10.253845 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:10.254153 (+ 308us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:10.254154 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:10.254196 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:10.254640 (+ 444us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 10:17:10.254644 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 10:17:10.254903 (+ 259us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 10:17:10.256398 (+ 1495us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 10:17:10.256399 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 10:17:10.256399 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 10:17:10.256628 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 10:17:10.256824 (+ 196us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 10:17:10.256825 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 10:17:10.256825 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 10:17:10.256887 (+ 62us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 10:17:10.257119 (+ 232us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 10:17:10.257120 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 10:17:10.257167 (+ 47us) client_negotiation.cc:769] Sending connection context
0619 10:17:10.257237 (+ 70us) client_negotiation.cc:240] Negotiation successful
0619 10:17:10.257340 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":89,"thread_start_us":62,"threads_started":1}
WARNING: no policy specified for kudu/127.0.206.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.206.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.206.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781864224577058-825-0/krb5kdc/kudu_127.0.206.67.keytab.
Entry for principal kudu/127.0.206.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781864224577058-825-0/krb5kdc/kudu_127.0.206.67.keytab.
I20260619 10:17:10.278416 825 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTU9OP6/build/release/bin/kudu
/tmp/dist-test-taskTU9OP6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781864224577058-825-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781864224577058-825-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781864224577058-825-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781864224577058-825-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.206.67:0
--local_ip_for_outbound_sockets=127.0.206.67
--webserver_interface=127.0.206.67
--webserver_port=0
--tserver_master_addrs=127.0.206.126:38679
--keytab_file=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781864224577058-825-0/krb5kdc/kudu_127.0.206.67.keytab
--principal=kudu/127.0.206.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.206.84:45993
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781864224577058-825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781864224577058-825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781864224577058-825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 10:17:10.357169 2289 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 10:17:10.357388 2289 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 10:17:10.357429 2289 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 10:17:10.358534 2289 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 10:17:10.358597 2289 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 10:17:10.358652 2289 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260619 10:17:10.358703 2289 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.206.67
I20260619 10:17:10.360105 2289 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.206.84:45993
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781864224577058-825-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781864224577058-825-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781864224577058-825-0/krb5kdc/kudu_127.0.206.67.keytab
--principal=kudu/127.0.206.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.206.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781864224577058-825-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.206.67
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.0.206.126:38679
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2289
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.206.67
--log_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781864224577058-825-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 09:32:44 UTC on ccycloud.quasar-gbtldp.root.comops.site
build id 4455440
I20260619 10:17:10.360347 2289 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 10:17:10.360554 2289 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 10:17:10.365463 2297 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 10:17:10.365456 2295 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 10:17:10.365463 2294 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 10:17:10.365731 2289 server_base.cc:1034] running on GCE node
I20260619 10:17:10.365948 2289 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 10:17:10.366130 2289 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 10:17:10.367295 2289 hybrid_clock.cc:648] HybridClock initialized: now 1781864230367276 us; error 30 us; skew 500 ppm
Jun 19 10:17:10 dist-test-slave-6pt2 krb5kdc[1892](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864230, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 10:17:10.369714 2289 init.cc:377] Logged in from keytab as kudu/127.0.206.67@KRBTEST.COM (short username kudu)
I20260619 10:17:10.370147 2289 webserver.cc:460] Webserver started at http://127.0.206.67:40093/ using document root <none> and password file <none>
I20260619 10:17:10.370360 2289 fs_manager.cc:359] Metadata directory not provided
I20260619 10:17:10.370427 2289 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 10:17:10.370573 2289 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 10:17:10.371681 2289 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781864224577058-825-0/minicluster-data/ts-2/data/instance:
uuid: "0c70e98c59624aeda6118cfda612fa22"
format_stamp: "Formatted at 2026-06-19 10:17:10 on dist-test-slave-6pt2"
server_key: "8ce7dd9044f5c443e4d7d73cc28d26fd"
server_key_iv: "7b2da100ed80fa79fae348d57170fa42"
server_key_version: "encryptionkey@0"
I20260619 10:17:10.372087 2289 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781864224577058-825-0/minicluster-data/ts-2/wal/instance:
uuid: "0c70e98c59624aeda6118cfda612fa22"
format_stamp: "Formatted at 2026-06-19 10:17:10 on dist-test-slave-6pt2"
server_key: "8ce7dd9044f5c443e4d7d73cc28d26fd"
server_key_iv: "7b2da100ed80fa79fae348d57170fa42"
server_key_version: "encryptionkey@0"
I20260619 10:17:10.373275 2289 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260619 10:17:10.373978 2304 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 10:17:10.374140 2289 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260619 10:17:10.374231 2289 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781864224577058-825-0/minicluster-data/ts-2/data,/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781864224577058-825-0/minicluster-data/ts-2/wal
uuid: "0c70e98c59624aeda6118cfda612fa22"
format_stamp: "Formatted at 2026-06-19 10:17:10 on dist-test-slave-6pt2"
server_key: "8ce7dd9044f5c443e4d7d73cc28d26fd"
server_key_iv: "7b2da100ed80fa79fae348d57170fa42"
server_key_version: "encryptionkey@0"
I20260619 10:17:10.374305 2289 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781864224577058-825-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781864224577058-825-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781864224577058-825-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 10:17:10.383019 2289 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 10:17:10.384684 2289 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 10:17:10.384843 2289 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 10:17:10.385294 2289 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 10:17:10.385356 2289 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 10:17:10.385388 2289 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 10:17:10.385416 2289 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 10:17:10.392150 2289 rpc_server.cc:307] RPC server started. Bound to: 127.0.206.67:37599
Jun 19 10:17:10 dist-test-slave-6pt2 krb5kdc[1892](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864230, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.67@KRBTEST.COM for kudu/127.0.206.126@KRBTEST.COM
I20260619 10:17:10.392220 2419 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.206.67:37599 every 8 connection(s)
I20260619 10:17:10.392481 2289 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781864224577058-825-0/minicluster-data/ts-2/data/info.pb
I20260619 10:17:10.392614 825 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTU9OP6/build/release/bin/kudu as pid 2289
I20260619 10:17:10.392710 825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781864224577058-825-0/minicluster-data/ts-2/wal/instance
I20260619 10:17:10.392891 825 external_mini_cluster.cc:1428] Setting key a6cdf7ba6edfee69cefdfd16e8a70cd7
I20260619 10:17:10.395582 2005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:10.385733 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.67:50771 (local address 127.0.206.126:38679)
0619 10:17:10.385806 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0619 10:17:10.385808 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 10:17:10.386002 (+ 194us) server_negotiation.cc:407] Connection header received
0619 10:17:10.386400 (+ 398us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:17:10.386402 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:17:10.386416 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:17:10.386492 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:17:10.387266 (+ 774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:10.388066 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:10.389709 (+ 1643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:10.390054 (+ 345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:10.390126 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:10.392819 (+ 2693us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 10:17:10.392820 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 10:17:10.392821 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 10:17:10.392829 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 10:17:10.394323 (+ 1494us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:17:10.394777 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:17:10.394778 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:17:10.394779 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:17:10.394828 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:17:10.395050 (+ 222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:17:10.395051 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:17:10.395051 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:17:10.395140 (+ 89us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 10:17:10.395184 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0619 10:17:10.395410 (+ 226us) server_negotiation.cc:299] Negotiation successful
0619 10:17:10.395488 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260619 10:17:10.395715 2313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:10.385683 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:38679 (local address 127.0.206.67:50771)
0619 10:17:10.385863 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0619 10:17:10.385899 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0619 10:17:10.386305 (+ 406us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 10:17:10.386598 (+ 293us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 10:17:10.386601 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 10:17:10.386928 (+ 327us) client_negotiation.cc:189] Negotiated authn=SASL
0619 10:17:10.387183 (+ 255us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:10.387190 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:10.388171 (+ 981us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:10.388172 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:10.389610 (+ 1438us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:10.389614 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:10.390150 (+ 536us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:10.390151 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:10.390291 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:10.390876 (+ 585us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 10:17:10.390886 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0619 10:17:10.392710 (+ 1824us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 10:17:10.394397 (+ 1687us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 10:17:10.394403 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 10:17:10.394405 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 10:17:10.394714 (+ 309us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 10:17:10.394909 (+ 195us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 10:17:10.394910 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 10:17:10.394910 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 10:17:10.395005 (+ 95us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 10:17:10.395190 (+ 185us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 10:17:10.395193 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 10:17:10.395291 (+ 98us) client_negotiation.cc:769] Sending connection context
0619 10:17:10.395375 (+ 84us) client_negotiation.cc:240] Negotiation successful
0619 10:17:10.395513 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":78,"threads_started":1}
I20260619 10:17:10.396178 2420 heartbeater.cc:347] Connected to a master server at 127.0.206.126:38679
I20260619 10:17:10.396250 2420 heartbeater.cc:464] Registering TS with master...
I20260619 10:17:10.396446 2420 heartbeater.cc:511] Master 127.0.206.126:38679 requested a full tablet report, sending...
I20260619 10:17:10.396756 1937 ts_manager.cc:194] Registered new tserver with Master: 0c70e98c59624aeda6118cfda612fa22 (127.0.206.67:37599)
I20260619 10:17:10.397197 1937 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.206.67@KRBTEST.COM'} at 127.0.206.67:50771
I20260619 10:17:10.401969 2005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:10.397189 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.67:57309 (local address 127.0.206.126:38679)
0619 10:17:10.397230 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0619 10:17:10.397231 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 10:17:10.397298 (+ 67us) server_negotiation.cc:407] Connection header received
0619 10:17:10.397390 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:17:10.397391 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:17:10.397400 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:17:10.397444 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:17:10.397808 (+ 364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:10.398250 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:10.398964 (+ 714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:10.399142 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:10.399213 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:10.399908 (+ 695us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 10:17:10.399909 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 10:17:10.399909 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 10:17:10.399915 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 10:17:10.401222 (+ 1307us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:17:10.401482 (+ 260us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:17:10.401483 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:17:10.401483 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:17:10.401526 (+ 43us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:17:10.401661 (+ 135us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:17:10.401661 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:17:10.401661 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:17:10.401736 (+ 75us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 10:17:10.401776 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0619 10:17:10.401870 (+ 94us) server_negotiation.cc:299] Negotiation successful
0619 10:17:10.401918 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260619 10:17:10.401975 2426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:10.397129 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:38679 (local address 127.0.206.67:57309)
0619 10:17:10.397234 (+ 105us) negotiation.cc:107] Waiting for socket to connect
0619 10:17:10.397243 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0619 10:17:10.397323 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 10:17:10.397457 (+ 134us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 10:17:10.397458 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 10:17:10.397669 (+ 211us) client_negotiation.cc:189] Negotiated authn=SASL
0619 10:17:10.397725 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:10.397727 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:10.398339 (+ 612us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:10.398340 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:10.398898 (+ 558us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:10.398899 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:10.399224 (+ 325us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:10.399224 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:10.399267 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:10.399591 (+ 324us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 10:17:10.399594 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0619 10:17:10.399837 (+ 243us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 10:17:10.401287 (+ 1450us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 10:17:10.401288 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 10:17:10.401288 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 10:17:10.401426 (+ 138us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 10:17:10.401579 (+ 153us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 10:17:10.401580 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 10:17:10.401580 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 10:17:10.401624 (+ 44us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 10:17:10.401779 (+ 155us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 10:17:10.401780 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 10:17:10.401809 (+ 29us) client_negotiation.cc:769] Sending connection context
0619 10:17:10.401849 (+ 40us) client_negotiation.cc:240] Negotiation successful
0619 10:17:10.401911 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":83,"thread_start_us":62,"threads_started":1}
I20260619 10:17:10.404419 825 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Jun 19 10:17:10 dist-test-slave-6pt2 krb5kdc[1892](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864230, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Jun 19 10:17:10 dist-test-slave-6pt2 krb5kdc[1892](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864230, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.0.206.126@KRBTEST.COM
I20260619 10:17:10.435981 2005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:10.427270 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38512 (local address 127.0.206.126:38679)
0619 10:17:10.427368 (+ 98us) server_negotiation.cc:206] Beginning negotiation
0619 10:17:10.427370 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 10:17:10.427381 (+ 11us) server_negotiation.cc:407] Connection header received
0619 10:17:10.427408 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:17:10.427410 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:17:10.427422 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:17:10.427493 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:17:10.428310 (+ 817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:10.429052 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:10.430350 (+ 1298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:10.430769 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:10.430854 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:10.433575 (+ 2721us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 10:17:10.433577 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 10:17:10.433578 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 10:17:10.433584 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 10:17:10.435050 (+ 1466us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:17:10.435343 (+ 293us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:17:10.435344 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:17:10.435344 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:17:10.435409 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:17:10.435577 (+ 168us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:17:10.435577 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:17:10.435577 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:17:10.435670 (+ 93us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 10:17:10.435736 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0619 10:17:10.435825 (+ 89us) server_negotiation.cc:299] Negotiation successful
0619 10:17:10.435881 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260619 10:17:10.437541 1937 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:38512:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 10:17:10.439204 1937 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 10:17:10.445384 2434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:10.441295 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.65:34017 (local address 127.0.0.1:52922)
0619 10:17:10.441447 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0619 10:17:10.441484 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0619 10:17:10.441564 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 10:17:10.441759 (+ 195us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 10:17:10.441762 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 10:17:10.441764 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 10:17:10.441825 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:10.441827 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:10.442922 (+ 1095us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:10.442923 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:10.444011 (+ 1088us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:10.444013 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:10.445053 (+ 1040us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:10.445054 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:10.445149 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:10.445161 (+ 12us) client_negotiation.cc:769] Sending connection context
0619 10:17:10.445237 (+ 76us) client_negotiation.cc:240] Negotiation successful
0619 10:17:10.445286 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":47,"threads_started":1}
I20260619 10:17:10.445574 2435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:10.441300 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.66:33487 (local address 127.0.0.1:49062)
0619 10:17:10.441730 (+ 430us) negotiation.cc:107] Waiting for socket to connect
0619 10:17:10.441740 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 10:17:10.441821 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 10:17:10.442069 (+ 248us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 10:17:10.442070 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 10:17:10.442073 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 10:17:10.442125 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:10.442126 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:10.443442 (+ 1316us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:10.443443 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:10.444393 (+ 950us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:10.444395 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:10.445225 (+ 830us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:10.445226 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:10.445279 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:10.445281 (+ 2us) client_negotiation.cc:769] Sending connection context
0619 10:17:10.445327 (+ 46us) client_negotiation.cc:240] Negotiation successful
0619 10:17:10.445358 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":417,"mutex_wait_us":4,"thread_start_us":54,"threads_started":1}
I20260619 10:17:10.445588 2439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:10.441526 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49062 (local address 127.0.206.66:33487)
0619 10:17:10.441697 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0619 10:17:10.441701 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 10:17:10.441880 (+ 179us) server_negotiation.cc:407] Connection header received
0619 10:17:10.441918 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:17:10.441920 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:17:10.441936 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:17:10.442007 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 10:17:10.442203 (+ 196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:10.443360 (+ 1157us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:10.444475 (+ 1115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:10.445043 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:10.445114 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:10.445190 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0619 10:17:10.445430 (+ 240us) server_negotiation.cc:299] Negotiation successful
0619 10:17:10.445480 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":146,"thread_start_us":81,"threads_started":1}
I20260619 10:17:10.445683 2436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:10.441301 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.67:37599 (local address 127.0.0.1:51564)
0619 10:17:10.441738 (+ 437us) negotiation.cc:107] Waiting for socket to connect
0619 10:17:10.441747 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0619 10:17:10.441801 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 10:17:10.442069 (+ 268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 10:17:10.442070 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 10:17:10.442071 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 10:17:10.442125 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:10.442127 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:10.443431 (+ 1304us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:10.443433 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:10.444447 (+ 1014us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:10.444449 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:10.445441 (+ 992us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:10.445442 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:10.445517 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:10.445521 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 10:17:10.445569 (+ 48us) client_negotiation.cc:240] Negotiation successful
0619 10:17:10.445602 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":425,"mutex_wait_us":6,"thread_start_us":59,"threads_started":1}
I20260619 10:17:10.445930 2438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:10.441513 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51564 (local address 127.0.206.67:37599)
0619 10:17:10.441683 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0619 10:17:10.441687 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 10:17:10.441874 (+ 187us) server_negotiation.cc:407] Connection header received
0619 10:17:10.441918 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:17:10.441920 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:17:10.441936 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:17:10.442007 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 10:17:10.442203 (+ 196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:10.443314 (+ 1111us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:10.444534 (+ 1220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:10.445273 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:10.445336 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:10.445404 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0619 10:17:10.445803 (+ 399us) server_negotiation.cc:299] Negotiation successful
0619 10:17:10.445839 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":139,"thread_start_us":74,"threads_started":1}
I20260619 10:17:10.445416 2437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:10.441377 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52922 (local address 127.0.206.65:34017)
0619 10:17:10.441556 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0619 10:17:10.441561 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 10:17:10.441572 (+ 11us) server_negotiation.cc:407] Connection header received
0619 10:17:10.441640 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:17:10.441643 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:17:10.441657 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:17:10.441755 (+ 98us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 10:17:10.441884 (+ 129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:10.442830 (+ 946us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:10.444174 (+ 1344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:10.444931 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:10.445004 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:10.445084 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0619 10:17:10.445265 (+ 181us) server_negotiation.cc:299] Negotiation successful
0619 10:17:10.445311 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":149,"thread_start_us":79,"threads_started":1}
I20260619 10:17:10.446331 2214 tablet_service.cc:1467] Processing CreateTablet for tablet f2a35f45743b494f81a7b2595b23f316 (DEFAULT_TABLE table=test-table [id=237c239fb03c4d738c2a70ee549e3b6a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 10:17:10.446460 2354 tablet_service.cc:1467] Processing CreateTablet for tablet f2a35f45743b494f81a7b2595b23f316 (DEFAULT_TABLE table=test-table [id=237c239fb03c4d738c2a70ee549e3b6a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 10:17:10.446735 2214 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f2a35f45743b494f81a7b2595b23f316. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 10:17:10.446893 2354 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f2a35f45743b494f81a7b2595b23f316. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 10:17:10.449440 2440 tablet_bootstrap.cc:492] T f2a35f45743b494f81a7b2595b23f316 P 0c70e98c59624aeda6118cfda612fa22: Bootstrap starting.
I20260619 10:17:10.449725 2441 tablet_bootstrap.cc:492] T f2a35f45743b494f81a7b2595b23f316 P 2bfba7e0bfad4fe4a249d625797d1669: Bootstrap starting.
I20260619 10:17:10.449925 2072 tablet_service.cc:1467] Processing CreateTablet for tablet f2a35f45743b494f81a7b2595b23f316 (DEFAULT_TABLE table=test-table [id=237c239fb03c4d738c2a70ee549e3b6a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 10:17:10.450085 2440 tablet_bootstrap.cc:654] T f2a35f45743b494f81a7b2595b23f316 P 0c70e98c59624aeda6118cfda612fa22: Neither blocks nor log segments found. Creating new log.
I20260619 10:17:10.450253 2072 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f2a35f45743b494f81a7b2595b23f316. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 10:17:10.450341 2441 tablet_bootstrap.cc:654] T f2a35f45743b494f81a7b2595b23f316 P 2bfba7e0bfad4fe4a249d625797d1669: Neither blocks nor log segments found. Creating new log.
I20260619 10:17:10.450392 2440 log.cc:824] T f2a35f45743b494f81a7b2595b23f316 P 0c70e98c59624aeda6118cfda612fa22: Log is configured to *not* fsync() on all Append() calls
I20260619 10:17:10.450692 2441 log.cc:824] T f2a35f45743b494f81a7b2595b23f316 P 2bfba7e0bfad4fe4a249d625797d1669: Log is configured to *not* fsync() on all Append() calls
I20260619 10:17:10.451267 2440 tablet_bootstrap.cc:492] T f2a35f45743b494f81a7b2595b23f316 P 0c70e98c59624aeda6118cfda612fa22: No bootstrap required, opened a new log
I20260619 10:17:10.451336 2440 ts_tablet_manager.cc:1397] T f2a35f45743b494f81a7b2595b23f316 P 0c70e98c59624aeda6118cfda612fa22: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260619 10:17:10.451395 2441 tablet_bootstrap.cc:492] T f2a35f45743b494f81a7b2595b23f316 P 2bfba7e0bfad4fe4a249d625797d1669: No bootstrap required, opened a new log
I20260619 10:17:10.451463 2441 ts_tablet_manager.cc:1397] T f2a35f45743b494f81a7b2595b23f316 P 2bfba7e0bfad4fe4a249d625797d1669: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260619 10:17:10.453091 2441 raft_consensus.cc:348] T f2a35f45743b494f81a7b2595b23f316 P 2bfba7e0bfad4fe4a249d625797d1669 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2bfba7e0bfad4fe4a249d625797d1669" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 33487 } } peers { permanent_uuid: "a38cfaebaa9a4ae8966419dc8abbd721" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 34017 } } peers { permanent_uuid: "0c70e98c59624aeda6118cfda612fa22" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 37599 } }
I20260619 10:17:10.453248 2441 raft_consensus.cc:374] T f2a35f45743b494f81a7b2595b23f316 P 2bfba7e0bfad4fe4a249d625797d1669 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 10:17:10.453289 2441 raft_consensus.cc:729] T f2a35f45743b494f81a7b2595b23f316 P 2bfba7e0bfad4fe4a249d625797d1669 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2bfba7e0bfad4fe4a249d625797d1669, State: Initialized, Role: FOLLOWER
I20260619 10:17:10.453393 2441 consensus_queue.cc:260] T f2a35f45743b494f81a7b2595b23f316 P 2bfba7e0bfad4fe4a249d625797d1669 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2bfba7e0bfad4fe4a249d625797d1669" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 33487 } } peers { permanent_uuid: "a38cfaebaa9a4ae8966419dc8abbd721" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 34017 } } peers { permanent_uuid: "0c70e98c59624aeda6118cfda612fa22" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 37599 } }
I20260619 10:17:10.453464 2440 raft_consensus.cc:348] T f2a35f45743b494f81a7b2595b23f316 P 0c70e98c59624aeda6118cfda612fa22 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2bfba7e0bfad4fe4a249d625797d1669" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 33487 } } peers { permanent_uuid: "a38cfaebaa9a4ae8966419dc8abbd721" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 34017 } } peers { permanent_uuid: "0c70e98c59624aeda6118cfda612fa22" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 37599 } }
I20260619 10:17:10.453629 2440 raft_consensus.cc:374] T f2a35f45743b494f81a7b2595b23f316 P 0c70e98c59624aeda6118cfda612fa22 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 10:17:10.453670 2441 ts_tablet_manager.cc:1428] T f2a35f45743b494f81a7b2595b23f316 P 2bfba7e0bfad4fe4a249d625797d1669: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260619 10:17:10.453677 2440 raft_consensus.cc:729] T f2a35f45743b494f81a7b2595b23f316 P 0c70e98c59624aeda6118cfda612fa22 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0c70e98c59624aeda6118cfda612fa22, State: Initialized, Role: FOLLOWER
I20260619 10:17:10.453794 2280 heartbeater.cc:503] Master 127.0.206.126:38679 was elected leader, sending a full tablet report...
I20260619 10:17:10.453788 2440 consensus_queue.cc:260] T f2a35f45743b494f81a7b2595b23f316 P 0c70e98c59624aeda6118cfda612fa22 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2bfba7e0bfad4fe4a249d625797d1669" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 33487 } } peers { permanent_uuid: "a38cfaebaa9a4ae8966419dc8abbd721" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 34017 } } peers { permanent_uuid: "0c70e98c59624aeda6118cfda612fa22" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 37599 } }
I20260619 10:17:10.454025 2440 ts_tablet_manager.cc:1428] T f2a35f45743b494f81a7b2595b23f316 P 0c70e98c59624aeda6118cfda612fa22: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260619 10:17:10.454056 2445 tablet_bootstrap.cc:492] T f2a35f45743b494f81a7b2595b23f316 P a38cfaebaa9a4ae8966419dc8abbd721: Bootstrap starting.
I20260619 10:17:10.454545 2420 heartbeater.cc:503] Master 127.0.206.126:38679 was elected leader, sending a full tablet report...
I20260619 10:17:10.454834 2445 tablet_bootstrap.cc:654] T f2a35f45743b494f81a7b2595b23f316 P a38cfaebaa9a4ae8966419dc8abbd721: Neither blocks nor log segments found. Creating new log.
I20260619 10:17:10.455189 2445 log.cc:824] T f2a35f45743b494f81a7b2595b23f316 P a38cfaebaa9a4ae8966419dc8abbd721: Log is configured to *not* fsync() on all Append() calls
I20260619 10:17:10.455915 2445 tablet_bootstrap.cc:492] T f2a35f45743b494f81a7b2595b23f316 P a38cfaebaa9a4ae8966419dc8abbd721: No bootstrap required, opened a new log
I20260619 10:17:10.455983 2445 ts_tablet_manager.cc:1397] T f2a35f45743b494f81a7b2595b23f316 P a38cfaebaa9a4ae8966419dc8abbd721: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260619 10:17:10.457379 2445 raft_consensus.cc:348] T f2a35f45743b494f81a7b2595b23f316 P a38cfaebaa9a4ae8966419dc8abbd721 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2bfba7e0bfad4fe4a249d625797d1669" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 33487 } } peers { permanent_uuid: "a38cfaebaa9a4ae8966419dc8abbd721" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 34017 } } peers { permanent_uuid: "0c70e98c59624aeda6118cfda612fa22" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 37599 } }
I20260619 10:17:10.457521 2445 raft_consensus.cc:374] T f2a35f45743b494f81a7b2595b23f316 P a38cfaebaa9a4ae8966419dc8abbd721 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 10:17:10.457568 2445 raft_consensus.cc:729] T f2a35f45743b494f81a7b2595b23f316 P a38cfaebaa9a4ae8966419dc8abbd721 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a38cfaebaa9a4ae8966419dc8abbd721, State: Initialized, Role: FOLLOWER
I20260619 10:17:10.457675 2445 consensus_queue.cc:260] T f2a35f45743b494f81a7b2595b23f316 P a38cfaebaa9a4ae8966419dc8abbd721 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2bfba7e0bfad4fe4a249d625797d1669" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 33487 } } peers { permanent_uuid: "a38cfaebaa9a4ae8966419dc8abbd721" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 34017 } } peers { permanent_uuid: "0c70e98c59624aeda6118cfda612fa22" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 37599 } }
I20260619 10:17:10.457940 2445 ts_tablet_manager.cc:1428] T f2a35f45743b494f81a7b2595b23f316 P a38cfaebaa9a4ae8966419dc8abbd721: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260619 10:17:10.457979 2140 heartbeater.cc:503] Master 127.0.206.126:38679 was elected leader, sending a full tablet report...
W20260619 10:17:10.499624 2281 tablet.cc:2369] T f2a35f45743b494f81a7b2595b23f316 P 2bfba7e0bfad4fe4a249d625797d1669: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 10:17:10.513319 2446 raft_consensus.cc:482] T f2a35f45743b494f81a7b2595b23f316 P 0c70e98c59624aeda6118cfda612fa22 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 10:17:10.513440 2446 raft_consensus.cc:504] T f2a35f45743b494f81a7b2595b23f316 P 0c70e98c59624aeda6118cfda612fa22 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2bfba7e0bfad4fe4a249d625797d1669" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 33487 } } peers { permanent_uuid: "a38cfaebaa9a4ae8966419dc8abbd721" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 34017 } } peers { permanent_uuid: "0c70e98c59624aeda6118cfda612fa22" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 37599 } }
I20260619 10:17:10.513839 2446 leader_election.cc:288] T f2a35f45743b494f81a7b2595b23f316 P 0c70e98c59624aeda6118cfda612fa22 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2bfba7e0bfad4fe4a249d625797d1669 (127.0.206.66:33487), a38cfaebaa9a4ae8966419dc8abbd721 (127.0.206.65:34017)
I20260619 10:17:10.517709 2437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:10.514140 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.67:57533 (local address 127.0.206.65:34017)
0619 10:17:10.514180 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0619 10:17:10.514182 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 10:17:10.514351 (+ 169us) server_negotiation.cc:407] Connection header received
0619 10:17:10.514386 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:17:10.514387 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:17:10.514394 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:17:10.514436 (+ 42us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 10:17:10.514611 (+ 175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:10.515303 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:10.516801 (+ 1498us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:10.517354 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:10.517419 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:10.517426 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0619 10:17:10.517603 (+ 177us) server_negotiation.cc:299] Negotiation successful
0619 10:17:10.517641 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260619 10:17:10.517709 2439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:10.514124 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.67:57187 (local address 127.0.206.66:33487)
0619 10:17:10.514170 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0619 10:17:10.514172 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 10:17:10.514185 (+ 13us) server_negotiation.cc:407] Connection header received
0619 10:17:10.514213 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:17:10.514214 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:17:10.514223 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:17:10.514287 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 10:17:10.514500 (+ 213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:10.515286 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:10.516769 (+ 1483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:10.517325 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:10.517400 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:10.517411 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0619 10:17:10.517603 (+ 192us) server_negotiation.cc:299] Negotiation successful
0619 10:17:10.517640 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260619 10:17:10.517733 2313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:10.514044 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.66:33487 (local address 127.0.206.67:57187)
0619 10:17:10.514104 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0619 10:17:10.514116 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 10:17:10.514159 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 10:17:10.514316 (+ 157us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 10:17:10.514317 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 10:17:10.514319 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 10:17:10.514390 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:10.514392 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:10.515375 (+ 983us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:10.515376 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:10.516660 (+ 1284us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:10.516663 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:10.517427 (+ 764us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:10.517428 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:10.517520 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:10.517525 (+ 5us) client_negotiation.cc:769] Sending connection context
0619 10:17:10.517582 (+ 57us) client_negotiation.cc:240] Negotiation successful
0619 10:17:10.517628 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":39,"mutex_wait_us":26}
I20260619 10:17:10.517740 2449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:10.514069 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.65:34017 (local address 127.0.206.67:57533)
0619 10:17:10.514240 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0619 10:17:10.514249 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0619 10:17:10.514330 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 10:17:10.514457 (+ 127us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 10:17:10.514458 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 10:17:10.514460 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 10:17:10.514518 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:10.514523 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:10.515375 (+ 852us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:10.515376 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:10.516660 (+ 1284us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:10.516663 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:10.517428 (+ 765us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:10.517429 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:10.517520 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:10.517525 (+ 5us) client_negotiation.cc:769] Sending connection context
0619 10:17:10.517582 (+ 57us) client_negotiation.cc:240] Negotiation successful
0619 10:17:10.517628 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":156,"mutex_wait_us":70,"thread_start_us":107,"threads_started":1}
I20260619 10:17:10.518146 2234 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f2a35f45743b494f81a7b2595b23f316" candidate_uuid: "0c70e98c59624aeda6118cfda612fa22" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2bfba7e0bfad4fe4a249d625797d1669" is_pre_election: true
I20260619 10:17:10.518308 2234 raft_consensus.cc:2454] T f2a35f45743b494f81a7b2595b23f316 P 2bfba7e0bfad4fe4a249d625797d1669 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0c70e98c59624aeda6118cfda612fa22 in term 0.
I20260619 10:17:10.518363 2094 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f2a35f45743b494f81a7b2595b23f316" candidate_uuid: "0c70e98c59624aeda6118cfda612fa22" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a38cfaebaa9a4ae8966419dc8abbd721" is_pre_election: true
I20260619 10:17:10.518525 2094 raft_consensus.cc:2454] T f2a35f45743b494f81a7b2595b23f316 P a38cfaebaa9a4ae8966419dc8abbd721 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0c70e98c59624aeda6118cfda612fa22 in term 0.
I20260619 10:17:10.518541 2307 leader_election.cc:302] T f2a35f45743b494f81a7b2595b23f316 P 0c70e98c59624aeda6118cfda612fa22 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0c70e98c59624aeda6118cfda612fa22, 2bfba7e0bfad4fe4a249d625797d1669; no voters:
I20260619 10:17:10.518680 2446 raft_consensus.cc:2781] T f2a35f45743b494f81a7b2595b23f316 P 0c70e98c59624aeda6118cfda612fa22 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 10:17:10.518734 2446 raft_consensus.cc:482] T f2a35f45743b494f81a7b2595b23f316 P 0c70e98c59624aeda6118cfda612fa22 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 10:17:10.518769 2446 raft_consensus.cc:3037] T f2a35f45743b494f81a7b2595b23f316 P 0c70e98c59624aeda6118cfda612fa22 [term 0 FOLLOWER]: Advancing to term 1
I20260619 10:17:10.519763 2446 raft_consensus.cc:504] T f2a35f45743b494f81a7b2595b23f316 P 0c70e98c59624aeda6118cfda612fa22 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2bfba7e0bfad4fe4a249d625797d1669" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 33487 } } peers { permanent_uuid: "a38cfaebaa9a4ae8966419dc8abbd721" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 34017 } } peers { permanent_uuid: "0c70e98c59624aeda6118cfda612fa22" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 37599 } }
I20260619 10:17:10.519907 2446 leader_election.cc:288] T f2a35f45743b494f81a7b2595b23f316 P 0c70e98c59624aeda6118cfda612fa22 [CANDIDATE]: Term 1 election: Requested vote from peers 2bfba7e0bfad4fe4a249d625797d1669 (127.0.206.66:33487), a38cfaebaa9a4ae8966419dc8abbd721 (127.0.206.65:34017)
I20260619 10:17:10.520066 2234 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f2a35f45743b494f81a7b2595b23f316" candidate_uuid: "0c70e98c59624aeda6118cfda612fa22" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2bfba7e0bfad4fe4a249d625797d1669"
I20260619 10:17:10.520064 2094 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f2a35f45743b494f81a7b2595b23f316" candidate_uuid: "0c70e98c59624aeda6118cfda612fa22" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a38cfaebaa9a4ae8966419dc8abbd721"
I20260619 10:17:10.520145 2094 raft_consensus.cc:3037] T f2a35f45743b494f81a7b2595b23f316 P a38cfaebaa9a4ae8966419dc8abbd721 [term 0 FOLLOWER]: Advancing to term 1
I20260619 10:17:10.520151 2234 raft_consensus.cc:3037] T f2a35f45743b494f81a7b2595b23f316 P 2bfba7e0bfad4fe4a249d625797d1669 [term 0 FOLLOWER]: Advancing to term 1
I20260619 10:17:10.521026 2094 raft_consensus.cc:2454] T f2a35f45743b494f81a7b2595b23f316 P a38cfaebaa9a4ae8966419dc8abbd721 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0c70e98c59624aeda6118cfda612fa22 in term 1.
I20260619 10:17:10.521229 2305 leader_election.cc:302] T f2a35f45743b494f81a7b2595b23f316 P 0c70e98c59624aeda6118cfda612fa22 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0c70e98c59624aeda6118cfda612fa22, a38cfaebaa9a4ae8966419dc8abbd721; no voters:
I20260619 10:17:10.521315 2446 raft_consensus.cc:2781] T f2a35f45743b494f81a7b2595b23f316 P 0c70e98c59624aeda6118cfda612fa22 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 10:17:10.521462 2446 raft_consensus.cc:686] T f2a35f45743b494f81a7b2595b23f316 P 0c70e98c59624aeda6118cfda612fa22 [term 1 LEADER]: Becoming Leader. State: Replica: 0c70e98c59624aeda6118cfda612fa22, State: Running, Role: LEADER
I20260619 10:17:10.521561 2446 consensus_queue.cc:237] T f2a35f45743b494f81a7b2595b23f316 P 0c70e98c59624aeda6118cfda612fa22 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2bfba7e0bfad4fe4a249d625797d1669" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 33487 } } peers { permanent_uuid: "a38cfaebaa9a4ae8966419dc8abbd721" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 34017 } } peers { permanent_uuid: "0c70e98c59624aeda6118cfda612fa22" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 37599 } }
I20260619 10:17:10.521673 2234 raft_consensus.cc:2454] T f2a35f45743b494f81a7b2595b23f316 P 2bfba7e0bfad4fe4a249d625797d1669 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0c70e98c59624aeda6118cfda612fa22 in term 1.
I20260619 10:17:10.522342 1937 catalog_manager.cc:5495] T f2a35f45743b494f81a7b2595b23f316 P 0c70e98c59624aeda6118cfda612fa22 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0c70e98c59624aeda6118cfda612fa22 (127.0.206.67). New cstate: current_term: 1 leader_uuid: "0c70e98c59624aeda6118cfda612fa22" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2bfba7e0bfad4fe4a249d625797d1669" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 33487 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a38cfaebaa9a4ae8966419dc8abbd721" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 34017 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0c70e98c59624aeda6118cfda612fa22" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 37599 } health_report { overall_health: HEALTHY } } }
I20260619 10:17:10.539928 2438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:10.537619 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51578 (local address 127.0.206.67:37599)
0619 10:17:10.537684 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0619 10:17:10.537686 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 10:17:10.537693 (+ 7us) server_negotiation.cc:407] Connection header received
0619 10:17:10.537763 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:17:10.537764 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:17:10.537778 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:17:10.537836 (+ 58us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 10:17:10.537989 (+ 153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:10.538416 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:10.539215 (+ 799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:10.539398 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:10.539451 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:10.539621 (+ 170us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 10:17:10.539703 (+ 82us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 10:17:10.539755 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0619 10:17:10.539799 (+ 44us) server_negotiation.cc:299] Negotiation successful
0619 10:17:10.539832 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260619 10:17:10.541478 2094 raft_consensus.cc:1261] T f2a35f45743b494f81a7b2595b23f316 P a38cfaebaa9a4ae8966419dc8abbd721 [term 1 FOLLOWER]: Refusing update from remote peer 0c70e98c59624aeda6118cfda612fa22: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 10:17:10.541554 2234 raft_consensus.cc:1261] T f2a35f45743b494f81a7b2595b23f316 P 2bfba7e0bfad4fe4a249d625797d1669 [term 1 FOLLOWER]: Refusing update from remote peer 0c70e98c59624aeda6118cfda612fa22: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 10:17:10.541795 2446 consensus_queue.cc:1035] T f2a35f45743b494f81a7b2595b23f316 P 0c70e98c59624aeda6118cfda612fa22 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a38cfaebaa9a4ae8966419dc8abbd721" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 34017 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 10:17:10.541908 2450 consensus_queue.cc:1035] T f2a35f45743b494f81a7b2595b23f316 P 0c70e98c59624aeda6118cfda612fa22 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2bfba7e0bfad4fe4a249d625797d1669" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 33487 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 10:17:10.544963 2455 mvcc.cc:205] Tried to move back new op lower bound from 7298515888295542784 to 7298515888216784896. Current Snapshot: MvccSnapshot[applied={T|T < 7298515888295542784 or (T in {7298515888295542784})}]
I20260619 10:17:10.547369 1936 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:38512:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 10:17:10.547461 1936 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:38512:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 10:17:10.548192 1936 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1f1f90ebcf834d238665f58d7af3e329: Sending DeleteTablet for 3 replicas of tablet f2a35f45743b494f81a7b2595b23f316
I20260619 10:17:10.548528 2072 tablet_service.cc:1514] Processing DeleteTablet for tablet f2a35f45743b494f81a7b2595b23f316 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 10:17:10 UTC) from {username='kudu', principal='kudu/127.0.206.126@KRBTEST.COM'} at 127.0.0.1:52922
I20260619 10:17:10.548537 2354 tablet_service.cc:1514] Processing DeleteTablet for tablet f2a35f45743b494f81a7b2595b23f316 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 10:17:10 UTC) from {username='kudu', principal='kudu/127.0.206.126@KRBTEST.COM'} at 127.0.0.1:51564
I20260619 10:17:10.548767 2214 tablet_service.cc:1514] Processing DeleteTablet for tablet f2a35f45743b494f81a7b2595b23f316 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 10:17:10 UTC) from {username='kudu', principal='kudu/127.0.206.126@KRBTEST.COM'} at 127.0.0.1:49062
I20260619 10:17:10.549088 2465 tablet_replica.cc:330] T f2a35f45743b494f81a7b2595b23f316 P 2bfba7e0bfad4fe4a249d625797d1669: stopping tablet replica
I20260619 10:17:10.549093 2464 tablet_replica.cc:330] T f2a35f45743b494f81a7b2595b23f316 P 0c70e98c59624aeda6118cfda612fa22: stopping tablet replica
I20260619 10:17:10.549237 2465 raft_consensus.cc:2229] T f2a35f45743b494f81a7b2595b23f316 P 2bfba7e0bfad4fe4a249d625797d1669 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 10:17:10.549220 2464 raft_consensus.cc:2229] T f2a35f45743b494f81a7b2595b23f316 P 0c70e98c59624aeda6118cfda612fa22 [term 1 LEADER]: Raft consensus shutting down.
I20260619 10:17:10.549325 2464 raft_consensus.cc:2258] T f2a35f45743b494f81a7b2595b23f316 P 0c70e98c59624aeda6118cfda612fa22 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 10:17:10.549433 2465 raft_consensus.cc:2258] T f2a35f45743b494f81a7b2595b23f316 P 2bfba7e0bfad4fe4a249d625797d1669 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 10:17:10.549980 2465 ts_tablet_manager.cc:1905] T f2a35f45743b494f81a7b2595b23f316 P 2bfba7e0bfad4fe4a249d625797d1669: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 10:17:10.550262 2464 ts_tablet_manager.cc:1905] T f2a35f45743b494f81a7b2595b23f316 P 0c70e98c59624aeda6118cfda612fa22: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 10:17:10.551671 1936 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:38512:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260619 10:17:10.551822 2464 ts_tablet_manager.cc:1918] T f2a35f45743b494f81a7b2595b23f316 P 0c70e98c59624aeda6118cfda612fa22: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
W20260619 10:17:10.551829 1936 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 10:17:10.551882 2464 log.cc:1196] T f2a35f45743b494f81a7b2595b23f316 P 0c70e98c59624aeda6118cfda612fa22: Deleting WAL directory at /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781864224577058-825-0/minicluster-data/ts-2/wal/wals/f2a35f45743b494f81a7b2595b23f316
I20260619 10:17:10.551771 2465 ts_tablet_manager.cc:1918] T f2a35f45743b494f81a7b2595b23f316 P 2bfba7e0bfad4fe4a249d625797d1669: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 10:17:10.552141 2464 ts_tablet_manager.cc:1939] T f2a35f45743b494f81a7b2595b23f316 P 0c70e98c59624aeda6118cfda612fa22: Deleting consensus metadata
I20260619 10:17:10.552147 2465 log.cc:1196] T f2a35f45743b494f81a7b2595b23f316 P 2bfba7e0bfad4fe4a249d625797d1669: Deleting WAL directory at /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781864224577058-825-0/minicluster-data/ts-1/wal/wals/f2a35f45743b494f81a7b2595b23f316
I20260619 10:17:10.552479 2465 ts_tablet_manager.cc:1939] T f2a35f45743b494f81a7b2595b23f316 P 2bfba7e0bfad4fe4a249d625797d1669: Deleting consensus metadata
I20260619 10:17:10.552497 1922 catalog_manager.cc:4841] TS 0c70e98c59624aeda6118cfda612fa22 (127.0.206.67:37599): tablet f2a35f45743b494f81a7b2595b23f316 (table test-table [id=237c239fb03c4d738c2a70ee549e3b6a]) successfully deleted
I20260619 10:17:10.552657 2463 tablet_replica.cc:330] T f2a35f45743b494f81a7b2595b23f316 P a38cfaebaa9a4ae8966419dc8abbd721: stopping tablet replica
I20260619 10:17:10.552743 2463 raft_consensus.cc:2229] T f2a35f45743b494f81a7b2595b23f316 P a38cfaebaa9a4ae8966419dc8abbd721 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 10:17:10.552843 2463 raft_consensus.cc:2258] T f2a35f45743b494f81a7b2595b23f316 P a38cfaebaa9a4ae8966419dc8abbd721 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 10:17:10.553234 2354 tablet_service.cc:1467] Processing CreateTablet for tablet 76e4c8cc9da447b3a9b6e74ecbd162a5 (DEFAULT_TABLE table=test-table [id=7c03db426678499ab0dfda29b4ea082c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 10:17:10.553297 2214 tablet_service.cc:1467] Processing CreateTablet for tablet 76e4c8cc9da447b3a9b6e74ecbd162a5 (DEFAULT_TABLE table=test-table [id=7c03db426678499ab0dfda29b4ea082c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 10:17:10.553380 2354 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 76e4c8cc9da447b3a9b6e74ecbd162a5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 10:17:10.553390 2214 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 76e4c8cc9da447b3a9b6e74ecbd162a5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 10:17:10.554167 2440 tablet_bootstrap.cc:492] T 76e4c8cc9da447b3a9b6e74ecbd162a5 P 0c70e98c59624aeda6118cfda612fa22: Bootstrap starting.
I20260619 10:17:10.554466 2441 tablet_bootstrap.cc:492] T 76e4c8cc9da447b3a9b6e74ecbd162a5 P 2bfba7e0bfad4fe4a249d625797d1669: Bootstrap starting.
I20260619 10:17:10.554510 2072 tablet_service.cc:1467] Processing CreateTablet for tablet 76e4c8cc9da447b3a9b6e74ecbd162a5 (DEFAULT_TABLE table=test-table [id=7c03db426678499ab0dfda29b4ea082c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 10:17:10.554566 2440 tablet_bootstrap.cc:654] T 76e4c8cc9da447b3a9b6e74ecbd162a5 P 0c70e98c59624aeda6118cfda612fa22: Neither blocks nor log segments found. Creating new log.
I20260619 10:17:10.554631 2072 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 76e4c8cc9da447b3a9b6e74ecbd162a5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 10:17:10.554883 2441 tablet_bootstrap.cc:654] T 76e4c8cc9da447b3a9b6e74ecbd162a5 P 2bfba7e0bfad4fe4a249d625797d1669: Neither blocks nor log segments found. Creating new log.
I20260619 10:17:10.555397 2441 tablet_bootstrap.cc:492] T 76e4c8cc9da447b3a9b6e74ecbd162a5 P 2bfba7e0bfad4fe4a249d625797d1669: No bootstrap required, opened a new log
I20260619 10:17:10.555454 2441 ts_tablet_manager.cc:1397] T 76e4c8cc9da447b3a9b6e74ecbd162a5 P 2bfba7e0bfad4fe4a249d625797d1669: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260619 10:17:10.555624 2441 raft_consensus.cc:348] T 76e4c8cc9da447b3a9b6e74ecbd162a5 P 2bfba7e0bfad4fe4a249d625797d1669 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a38cfaebaa9a4ae8966419dc8abbd721" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 34017 } } peers { permanent_uuid: "2bfba7e0bfad4fe4a249d625797d1669" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 33487 } } peers { permanent_uuid: "0c70e98c59624aeda6118cfda612fa22" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 37599 } }
I20260619 10:17:10.555688 2441 raft_consensus.cc:374] T 76e4c8cc9da447b3a9b6e74ecbd162a5 P 2bfba7e0bfad4fe4a249d625797d1669 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 10:17:10.555713 2441 raft_consensus.cc:729] T 76e4c8cc9da447b3a9b6e74ecbd162a5 P 2bfba7e0bfad4fe4a249d625797d1669 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2bfba7e0bfad4fe4a249d625797d1669, State: Initialized, Role: FOLLOWER
I20260619 10:17:10.555768 2441 consensus_queue.cc:260] T 76e4c8cc9da447b3a9b6e74ecbd162a5 P 2bfba7e0bfad4fe4a249d625797d1669 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a38cfaebaa9a4ae8966419dc8abbd721" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 34017 } } peers { permanent_uuid: "2bfba7e0bfad4fe4a249d625797d1669" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 33487 } } peers { permanent_uuid: "0c70e98c59624aeda6118cfda612fa22" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 37599 } }
I20260619 10:17:10.555852 2441 ts_tablet_manager.cc:1428] T 76e4c8cc9da447b3a9b6e74ecbd162a5 P 2bfba7e0bfad4fe4a249d625797d1669: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260619 10:17:10.556607 2445 tablet_bootstrap.cc:492] T 76e4c8cc9da447b3a9b6e74ecbd162a5 P a38cfaebaa9a4ae8966419dc8abbd721: Bootstrap starting.
I20260619 10:17:10.556625 1923 catalog_manager.cc:4841] TS 2bfba7e0bfad4fe4a249d625797d1669 (127.0.206.66:33487): tablet f2a35f45743b494f81a7b2595b23f316 (table test-table [id=237c239fb03c4d738c2a70ee549e3b6a]) successfully deleted
I20260619 10:17:10.557034 2445 tablet_bootstrap.cc:654] T 76e4c8cc9da447b3a9b6e74ecbd162a5 P a38cfaebaa9a4ae8966419dc8abbd721: Neither blocks nor log segments found. Creating new log.
I20260619 10:17:10.557096 2463 ts_tablet_manager.cc:1905] T f2a35f45743b494f81a7b2595b23f316 P a38cfaebaa9a4ae8966419dc8abbd721: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 10:17:10.557613 2445 tablet_bootstrap.cc:492] T 76e4c8cc9da447b3a9b6e74ecbd162a5 P a38cfaebaa9a4ae8966419dc8abbd721: No bootstrap required, opened a new log
I20260619 10:17:10.557672 2445 ts_tablet_manager.cc:1397] T 76e4c8cc9da447b3a9b6e74ecbd162a5 P a38cfaebaa9a4ae8966419dc8abbd721: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260619 10:17:10.557823 2445 raft_consensus.cc:348] T 76e4c8cc9da447b3a9b6e74ecbd162a5 P a38cfaebaa9a4ae8966419dc8abbd721 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a38cfaebaa9a4ae8966419dc8abbd721" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 34017 } } peers { permanent_uuid: "2bfba7e0bfad4fe4a249d625797d1669" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 33487 } } peers { permanent_uuid: "0c70e98c59624aeda6118cfda612fa22" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 37599 } }
I20260619 10:17:10.557899 2445 raft_consensus.cc:374] T 76e4c8cc9da447b3a9b6e74ecbd162a5 P a38cfaebaa9a4ae8966419dc8abbd721 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 10:17:10.557927 2445 raft_consensus.cc:729] T 76e4c8cc9da447b3a9b6e74ecbd162a5 P a38cfaebaa9a4ae8966419dc8abbd721 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a38cfaebaa9a4ae8966419dc8abbd721, State: Initialized, Role: FOLLOWER
I20260619 10:17:10.557971 2445 consensus_queue.cc:260] T 76e4c8cc9da447b3a9b6e74ecbd162a5 P a38cfaebaa9a4ae8966419dc8abbd721 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a38cfaebaa9a4ae8966419dc8abbd721" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 34017 } } peers { permanent_uuid: "2bfba7e0bfad4fe4a249d625797d1669" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 33487 } } peers { permanent_uuid: "0c70e98c59624aeda6118cfda612fa22" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 37599 } }
I20260619 10:17:10.558053 2445 ts_tablet_manager.cc:1428] T 76e4c8cc9da447b3a9b6e74ecbd162a5 P a38cfaebaa9a4ae8966419dc8abbd721: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260619 10:17:10.558436 2463 ts_tablet_manager.cc:1918] T f2a35f45743b494f81a7b2595b23f316 P a38cfaebaa9a4ae8966419dc8abbd721: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 10:17:10.558511 2463 log.cc:1196] T f2a35f45743b494f81a7b2595b23f316 P a38cfaebaa9a4ae8966419dc8abbd721: Deleting WAL directory at /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781864224577058-825-0/minicluster-data/ts-0/wal/wals/f2a35f45743b494f81a7b2595b23f316
I20260619 10:17:10.558768 2463 ts_tablet_manager.cc:1939] T f2a35f45743b494f81a7b2595b23f316 P a38cfaebaa9a4ae8966419dc8abbd721: Deleting consensus metadata
I20260619 10:17:10.559146 1921 catalog_manager.cc:4841] TS a38cfaebaa9a4ae8966419dc8abbd721 (127.0.206.65:34017): tablet f2a35f45743b494f81a7b2595b23f316 (table test-table [id=237c239fb03c4d738c2a70ee549e3b6a]) successfully deleted
I20260619 10:17:10.559378 2440 tablet_bootstrap.cc:492] T 76e4c8cc9da447b3a9b6e74ecbd162a5 P 0c70e98c59624aeda6118cfda612fa22: No bootstrap required, opened a new log
I20260619 10:17:10.559444 2440 ts_tablet_manager.cc:1397] T 76e4c8cc9da447b3a9b6e74ecbd162a5 P 0c70e98c59624aeda6118cfda612fa22: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.000s
I20260619 10:17:10.559577 2440 raft_consensus.cc:348] T 76e4c8cc9da447b3a9b6e74ecbd162a5 P 0c70e98c59624aeda6118cfda612fa22 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a38cfaebaa9a4ae8966419dc8abbd721" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 34017 } } peers { permanent_uuid: "2bfba7e0bfad4fe4a249d625797d1669" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 33487 } } peers { permanent_uuid: "0c70e98c59624aeda6118cfda612fa22" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 37599 } }
I20260619 10:17:10.559646 2440 raft_consensus.cc:374] T 76e4c8cc9da447b3a9b6e74ecbd162a5 P 0c70e98c59624aeda6118cfda612fa22 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 10:17:10.559667 2440 raft_consensus.cc:729] T 76e4c8cc9da447b3a9b6e74ecbd162a5 P 0c70e98c59624aeda6118cfda612fa22 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0c70e98c59624aeda6118cfda612fa22, State: Initialized, Role: FOLLOWER
I20260619 10:17:10.559710 2440 consensus_queue.cc:260] T 76e4c8cc9da447b3a9b6e74ecbd162a5 P 0c70e98c59624aeda6118cfda612fa22 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a38cfaebaa9a4ae8966419dc8abbd721" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 34017 } } peers { permanent_uuid: "2bfba7e0bfad4fe4a249d625797d1669" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 33487 } } peers { permanent_uuid: "0c70e98c59624aeda6118cfda612fa22" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 37599 } }
I20260619 10:17:10.559783 2440 ts_tablet_manager.cc:1428] T 76e4c8cc9da447b3a9b6e74ecbd162a5 P 0c70e98c59624aeda6118cfda612fa22: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
W20260619 10:17:10.602015 2141 tablet.cc:2369] T 76e4c8cc9da447b3a9b6e74ecbd162a5 P a38cfaebaa9a4ae8966419dc8abbd721: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 10:17:10.617560 2444 raft_consensus.cc:482] T 76e4c8cc9da447b3a9b6e74ecbd162a5 P 2bfba7e0bfad4fe4a249d625797d1669 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 10:17:10.617660 2444 raft_consensus.cc:504] T 76e4c8cc9da447b3a9b6e74ecbd162a5 P 2bfba7e0bfad4fe4a249d625797d1669 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a38cfaebaa9a4ae8966419dc8abbd721" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 34017 } } peers { permanent_uuid: "2bfba7e0bfad4fe4a249d625797d1669" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 33487 } } peers { permanent_uuid: "0c70e98c59624aeda6118cfda612fa22" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 37599 } }
I20260619 10:17:10.618049 2444 leader_election.cc:288] T 76e4c8cc9da447b3a9b6e74ecbd162a5 P 2bfba7e0bfad4fe4a249d625797d1669 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a38cfaebaa9a4ae8966419dc8abbd721 (127.0.206.65:34017), 0c70e98c59624aeda6118cfda612fa22 (127.0.206.67:37599)
I20260619 10:17:10.621410 2437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:10.618532 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.66:54321 (local address 127.0.206.65:34017)
0619 10:17:10.618623 (+ 91us) server_negotiation.cc:206] Beginning negotiation
0619 10:17:10.618625 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 10:17:10.618634 (+ 9us) server_negotiation.cc:407] Connection header received
0619 10:17:10.618660 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:17:10.618661 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:17:10.618670 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:17:10.618724 (+ 54us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 10:17:10.618878 (+ 154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:10.619686 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:10.620681 (+ 995us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:10.621078 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:10.621148 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:10.621156 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0619 10:17:10.621307 (+ 151us) server_negotiation.cc:299] Negotiation successful
0619 10:17:10.621343 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260619 10:17:10.621425 2173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:10.618202 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.65:34017 (local address 127.0.206.66:54321)
0619 10:17:10.618266 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0619 10:17:10.618278 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 10:17:10.618336 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 10:17:10.618726 (+ 390us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 10:17:10.618726 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 10:17:10.618728 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 10:17:10.618801 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:10.618802 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:10.619742 (+ 940us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:10.619743 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:10.620580 (+ 837us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:10.620582 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:10.621155 (+ 573us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:10.621156 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:10.621229 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:10.621233 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 10:17:10.621294 (+ 61us) client_negotiation.cc:240] Negotiation successful
0619 10:17:10.621334 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":44,"mutex_wait_us":43}
I20260619 10:17:10.621683 2094 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "76e4c8cc9da447b3a9b6e74ecbd162a5" candidate_uuid: "2bfba7e0bfad4fe4a249d625797d1669" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a38cfaebaa9a4ae8966419dc8abbd721" is_pre_election: true
I20260619 10:17:10.621788 2094 raft_consensus.cc:2454] T 76e4c8cc9da447b3a9b6e74ecbd162a5 P a38cfaebaa9a4ae8966419dc8abbd721 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2bfba7e0bfad4fe4a249d625797d1669 in term 0.
I20260619 10:17:10.621950 2438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:10.618275 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.66:38045 (local address 127.0.206.67:37599)
0619 10:17:10.618318 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0619 10:17:10.618320 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 10:17:10.618589 (+ 269us) server_negotiation.cc:407] Connection header received
0619 10:17:10.618677 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:17:10.618678 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:17:10.618688 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:17:10.618743 (+ 55us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 10:17:10.618948 (+ 205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:10.619640 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:10.621051 (+ 1411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:10.621597 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:10.621677 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:10.621684 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0619 10:17:10.621850 (+ 166us) server_negotiation.cc:299] Negotiation successful
0619 10:17:10.621889 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260619 10:17:10.621937 2165 leader_election.cc:302] T 76e4c8cc9da447b3a9b6e74ecbd162a5 P 2bfba7e0bfad4fe4a249d625797d1669 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2bfba7e0bfad4fe4a249d625797d1669, a38cfaebaa9a4ae8966419dc8abbd721; no voters:
I20260619 10:17:10.622089 2444 raft_consensus.cc:2781] T 76e4c8cc9da447b3a9b6e74ecbd162a5 P 2bfba7e0bfad4fe4a249d625797d1669 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 10:17:10.622139 2444 raft_consensus.cc:482] T 76e4c8cc9da447b3a9b6e74ecbd162a5 P 2bfba7e0bfad4fe4a249d625797d1669 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 10:17:10.622166 2444 raft_consensus.cc:3037] T 76e4c8cc9da447b3a9b6e74ecbd162a5 P 2bfba7e0bfad4fe4a249d625797d1669 [term 0 FOLLOWER]: Advancing to term 1
I20260619 10:17:10.622195 2469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:10.618203 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.67:37599 (local address 127.0.206.66:38045)
0619 10:17:10.618508 (+ 305us) negotiation.cc:107] Waiting for socket to connect
0619 10:17:10.618518 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 10:17:10.618618 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 10:17:10.618794 (+ 176us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 10:17:10.618795 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 10:17:10.618798 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 10:17:10.618884 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:10.618886 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:10.619723 (+ 837us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:10.619724 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:10.620959 (+ 1235us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:10.620962 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:10.621679 (+ 717us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:10.621680 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:10.621759 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:10.621764 (+ 5us) client_negotiation.cc:769] Sending connection context
0619 10:17:10.622023 (+ 259us) client_negotiation.cc:240] Negotiation successful
0619 10:17:10.622062 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":292,"mutex_wait_us":207,"thread_start_us":86,"threads_started":1}
I20260619 10:17:10.622965 2444 raft_consensus.cc:504] T 76e4c8cc9da447b3a9b6e74ecbd162a5 P 2bfba7e0bfad4fe4a249d625797d1669 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a38cfaebaa9a4ae8966419dc8abbd721" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 34017 } } peers { permanent_uuid: "2bfba7e0bfad4fe4a249d625797d1669" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 33487 } } peers { permanent_uuid: "0c70e98c59624aeda6118cfda612fa22" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 37599 } }
I20260619 10:17:10.623109 2444 leader_election.cc:288] T 76e4c8cc9da447b3a9b6e74ecbd162a5 P 2bfba7e0bfad4fe4a249d625797d1669 [CANDIDATE]: Term 1 election: Requested vote from peers a38cfaebaa9a4ae8966419dc8abbd721 (127.0.206.65:34017), 0c70e98c59624aeda6118cfda612fa22 (127.0.206.67:37599)
I20260619 10:17:10.623291 2094 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "76e4c8cc9da447b3a9b6e74ecbd162a5" candidate_uuid: "2bfba7e0bfad4fe4a249d625797d1669" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a38cfaebaa9a4ae8966419dc8abbd721"
I20260619 10:17:10.623368 2094 raft_consensus.cc:3037] T 76e4c8cc9da447b3a9b6e74ecbd162a5 P a38cfaebaa9a4ae8966419dc8abbd721 [term 0 FOLLOWER]: Advancing to term 1
I20260619 10:17:10.623432 2374 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "76e4c8cc9da447b3a9b6e74ecbd162a5" candidate_uuid: "2bfba7e0bfad4fe4a249d625797d1669" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0c70e98c59624aeda6118cfda612fa22" is_pre_election: true
I20260619 10:17:10.623438 2373 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "76e4c8cc9da447b3a9b6e74ecbd162a5" candidate_uuid: "2bfba7e0bfad4fe4a249d625797d1669" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0c70e98c59624aeda6118cfda612fa22"
I20260619 10:17:10.623600 2374 raft_consensus.cc:2454] T 76e4c8cc9da447b3a9b6e74ecbd162a5 P 0c70e98c59624aeda6118cfda612fa22 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2bfba7e0bfad4fe4a249d625797d1669 in term 0.
I20260619 10:17:10.624184 2094 raft_consensus.cc:2454] T 76e4c8cc9da447b3a9b6e74ecbd162a5 P a38cfaebaa9a4ae8966419dc8abbd721 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2bfba7e0bfad4fe4a249d625797d1669 in term 1.
I20260619 10:17:10.624341 2165 leader_election.cc:302] T 76e4c8cc9da447b3a9b6e74ecbd162a5 P 2bfba7e0bfad4fe4a249d625797d1669 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2bfba7e0bfad4fe4a249d625797d1669, a38cfaebaa9a4ae8966419dc8abbd721; no voters: 0c70e98c59624aeda6118cfda612fa22
I20260619 10:17:10.624459 2444 raft_consensus.cc:2781] T 76e4c8cc9da447b3a9b6e74ecbd162a5 P 2bfba7e0bfad4fe4a249d625797d1669 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 10:17:10.624629 2444 raft_consensus.cc:686] T 76e4c8cc9da447b3a9b6e74ecbd162a5 P 2bfba7e0bfad4fe4a249d625797d1669 [term 1 LEADER]: Becoming Leader. State: Replica: 2bfba7e0bfad4fe4a249d625797d1669, State: Running, Role: LEADER
I20260619 10:17:10.624697 2444 consensus_queue.cc:237] T 76e4c8cc9da447b3a9b6e74ecbd162a5 P 2bfba7e0bfad4fe4a249d625797d1669 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a38cfaebaa9a4ae8966419dc8abbd721" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 34017 } } peers { permanent_uuid: "2bfba7e0bfad4fe4a249d625797d1669" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 33487 } } peers { permanent_uuid: "0c70e98c59624aeda6118cfda612fa22" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 37599 } }
I20260619 10:17:10.625342 1937 catalog_manager.cc:5495] T 76e4c8cc9da447b3a9b6e74ecbd162a5 P 2bfba7e0bfad4fe4a249d625797d1669 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2bfba7e0bfad4fe4a249d625797d1669 (127.0.206.66). New cstate: current_term: 1 leader_uuid: "2bfba7e0bfad4fe4a249d625797d1669" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a38cfaebaa9a4ae8966419dc8abbd721" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 34017 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2bfba7e0bfad4fe4a249d625797d1669" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 33487 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0c70e98c59624aeda6118cfda612fa22" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 37599 } health_report { overall_health: UNKNOWN } } }
W20260619 10:17:10.643421 2421 tablet.cc:2369] T 76e4c8cc9da447b3a9b6e74ecbd162a5 P 0c70e98c59624aeda6118cfda612fa22: Can't schedule compaction. Clean time has not been advanced past its initial value.
Jun 19 10:17:10 dist-test-slave-6pt2 krb5kdc[1892](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864229, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.126@KRBTEST.COM for kudu/127.0.206.126@KRBTEST.COM
I20260619 10:17:10.651051 2477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:10.643314 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:38679 (local address 127.0.0.1:38518)
0619 10:17:10.643496 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0619 10:17:10.643507 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 10:17:10.643584 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 10:17:10.643748 (+ 164us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 10:17:10.643749 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 10:17:10.644020 (+ 271us) client_negotiation.cc:189] Negotiated authn=SASL
0619 10:17:10.644061 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:10.644062 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:10.644971 (+ 909us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:10.644972 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:10.645465 (+ 493us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:10.645466 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:10.645889 (+ 423us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:10.645890 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:10.645925 (+ 35us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:10.646447 (+ 522us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 10:17:10.646451 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 10:17:10.647916 (+ 1465us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 10:17:10.649998 (+ 2082us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 10:17:10.650003 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 10:17:10.650005 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 10:17:10.650205 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 10:17:10.650402 (+ 197us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 10:17:10.650403 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 10:17:10.650403 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 10:17:10.650493 (+ 90us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 10:17:10.650791 (+ 298us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 10:17:10.650796 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 10:17:10.650844 (+ 48us) client_negotiation.cc:769] Sending connection context
0619 10:17:10.650893 (+ 49us) client_negotiation.cc:240] Negotiation successful
0619 10:17:10.650977 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":81,"threads_started":1}
I20260619 10:17:10.651147 2005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:10.643392 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38518 (local address 127.0.206.126:38679)
0619 10:17:10.643434 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0619 10:17:10.643436 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 10:17:10.643559 (+ 123us) server_negotiation.cc:407] Connection header received
0619 10:17:10.643656 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:17:10.643658 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:17:10.643669 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:17:10.643735 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:17:10.644144 (+ 409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:10.644888 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:10.645544 (+ 656us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:10.645817 (+ 273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:10.645885 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:10.648007 (+ 2122us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 10:17:10.648008 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 10:17:10.648008 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 10:17:10.648016 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 10:17:10.649911 (+ 1895us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:17:10.650275 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:17:10.650276 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:17:10.650277 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:17:10.650336 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:17:10.650557 (+ 221us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:17:10.650558 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:17:10.650558 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:17:10.650702 (+ 144us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 10:17:10.650783 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0619 10:17:10.650923 (+ 140us) server_negotiation.cc:299] Negotiation successful
0619 10:17:10.651041 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":38,"server-negotiator.queue_time_us":22}
I20260619 10:17:10.653029 1937 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.0.206.126@KRBTEST.COM'} at 127.0.0.1:38518:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260619 10:17:10.653246 1937 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 10:17:10.654663 2072 tablet_service.cc:1467] Processing CreateTablet for tablet 11769cdadc70416c8459a6f4b6850276 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=e62864a3bdab429fbde43cd9624fa842]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260619 10:17:10.654675 2354 tablet_service.cc:1467] Processing CreateTablet for tablet 11769cdadc70416c8459a6f4b6850276 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=e62864a3bdab429fbde43cd9624fa842]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260619 10:17:10.654675 2214 tablet_service.cc:1467] Processing CreateTablet for tablet 11769cdadc70416c8459a6f4b6850276 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=e62864a3bdab429fbde43cd9624fa842]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260619 10:17:10.654815 2072 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 11769cdadc70416c8459a6f4b6850276. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 10:17:10.654815 2214 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 11769cdadc70416c8459a6f4b6850276. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 10:17:10.654825 2354 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 11769cdadc70416c8459a6f4b6850276. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 10:17:10.656288 2440 tablet_bootstrap.cc:492] T 11769cdadc70416c8459a6f4b6850276 P 0c70e98c59624aeda6118cfda612fa22: Bootstrap starting.
I20260619 10:17:10.656632 2440 tablet_bootstrap.cc:654] T 11769cdadc70416c8459a6f4b6850276 P 0c70e98c59624aeda6118cfda612fa22: Neither blocks nor log segments found. Creating new log.
I20260619 10:17:10.656920 2441 tablet_bootstrap.cc:492] T 11769cdadc70416c8459a6f4b6850276 P 2bfba7e0bfad4fe4a249d625797d1669: Bootstrap starting.
I20260619 10:17:10.657238 2440 tablet_bootstrap.cc:492] T 11769cdadc70416c8459a6f4b6850276 P 0c70e98c59624aeda6118cfda612fa22: No bootstrap required, opened a new log
I20260619 10:17:10.657269 2441 tablet_bootstrap.cc:654] T 11769cdadc70416c8459a6f4b6850276 P 2bfba7e0bfad4fe4a249d625797d1669: Neither blocks nor log segments found. Creating new log.
I20260619 10:17:10.657279 2440 ts_tablet_manager.cc:1397] T 11769cdadc70416c8459a6f4b6850276 P 0c70e98c59624aeda6118cfda612fa22: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260619 10:17:10.657421 2440 raft_consensus.cc:348] T 11769cdadc70416c8459a6f4b6850276 P 0c70e98c59624aeda6118cfda612fa22 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a38cfaebaa9a4ae8966419dc8abbd721" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 34017 } } peers { permanent_uuid: "2bfba7e0bfad4fe4a249d625797d1669" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 33487 } } peers { permanent_uuid: "0c70e98c59624aeda6118cfda612fa22" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 37599 } }
I20260619 10:17:10.657491 2440 raft_consensus.cc:374] T 11769cdadc70416c8459a6f4b6850276 P 0c70e98c59624aeda6118cfda612fa22 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 10:17:10.657510 2440 raft_consensus.cc:729] T 11769cdadc70416c8459a6f4b6850276 P 0c70e98c59624aeda6118cfda612fa22 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0c70e98c59624aeda6118cfda612fa22, State: Initialized, Role: FOLLOWER
I20260619 10:17:10.657552 2440 consensus_queue.cc:260] T 11769cdadc70416c8459a6f4b6850276 P 0c70e98c59624aeda6118cfda612fa22 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a38cfaebaa9a4ae8966419dc8abbd721" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 34017 } } peers { permanent_uuid: "2bfba7e0bfad4fe4a249d625797d1669" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 33487 } } peers { permanent_uuid: "0c70e98c59624aeda6118cfda612fa22" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 37599 } }
I20260619 10:17:10.657557 2445 tablet_bootstrap.cc:492] T 11769cdadc70416c8459a6f4b6850276 P a38cfaebaa9a4ae8966419dc8abbd721: Bootstrap starting.
I20260619 10:17:10.657624 2440 ts_tablet_manager.cc:1428] T 11769cdadc70416c8459a6f4b6850276 P 0c70e98c59624aeda6118cfda612fa22: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260619 10:17:10.657723 2460 tablet_replica.cc:439] T 11769cdadc70416c8459a6f4b6850276 P 0c70e98c59624aeda6118cfda612fa22: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a38cfaebaa9a4ae8966419dc8abbd721" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 34017 } } peers { permanent_uuid: "2bfba7e0bfad4fe4a249d625797d1669" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 33487 } } peers { permanent_uuid: "0c70e98c59624aeda6118cfda612fa22" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 37599 } } }
I20260619 10:17:10.657770 2460 tablet_replica.cc:442] T 11769cdadc70416c8459a6f4b6850276 P 0c70e98c59624aeda6118cfda612fa22: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 10:17:10.657969 2445 tablet_bootstrap.cc:654] T 11769cdadc70416c8459a6f4b6850276 P a38cfaebaa9a4ae8966419dc8abbd721: Neither blocks nor log segments found. Creating new log.
I20260619 10:17:10.658818 2441 tablet_bootstrap.cc:492] T 11769cdadc70416c8459a6f4b6850276 P 2bfba7e0bfad4fe4a249d625797d1669: No bootstrap required, opened a new log
I20260619 10:17:10.658872 2441 ts_tablet_manager.cc:1397] T 11769cdadc70416c8459a6f4b6850276 P 2bfba7e0bfad4fe4a249d625797d1669: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260619 10:17:10.659150 2445 tablet_bootstrap.cc:492] T 11769cdadc70416c8459a6f4b6850276 P a38cfaebaa9a4ae8966419dc8abbd721: No bootstrap required, opened a new log
I20260619 10:17:10.659150 2441 raft_consensus.cc:348] T 11769cdadc70416c8459a6f4b6850276 P 2bfba7e0bfad4fe4a249d625797d1669 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a38cfaebaa9a4ae8966419dc8abbd721" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 34017 } } peers { permanent_uuid: "2bfba7e0bfad4fe4a249d625797d1669" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 33487 } } peers { permanent_uuid: "0c70e98c59624aeda6118cfda612fa22" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 37599 } }
I20260619 10:17:10.659209 2441 raft_consensus.cc:374] T 11769cdadc70416c8459a6f4b6850276 P 2bfba7e0bfad4fe4a249d625797d1669 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 10:17:10.659226 2441 raft_consensus.cc:729] T 11769cdadc70416c8459a6f4b6850276 P 2bfba7e0bfad4fe4a249d625797d1669 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2bfba7e0bfad4fe4a249d625797d1669, State: Initialized, Role: FOLLOWER
I20260619 10:17:10.659216 2445 ts_tablet_manager.cc:1397] T 11769cdadc70416c8459a6f4b6850276 P a38cfaebaa9a4ae8966419dc8abbd721: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260619 10:17:10.659268 2441 consensus_queue.cc:260] T 11769cdadc70416c8459a6f4b6850276 P 2bfba7e0bfad4fe4a249d625797d1669 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a38cfaebaa9a4ae8966419dc8abbd721" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 34017 } } peers { permanent_uuid: "2bfba7e0bfad4fe4a249d625797d1669" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 33487 } } peers { permanent_uuid: "0c70e98c59624aeda6118cfda612fa22" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 37599 } }
I20260619 10:17:10.659341 2441 ts_tablet_manager.cc:1428] T 11769cdadc70416c8459a6f4b6850276 P 2bfba7e0bfad4fe4a249d625797d1669: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260619 10:17:10.659385 2445 raft_consensus.cc:348] T 11769cdadc70416c8459a6f4b6850276 P a38cfaebaa9a4ae8966419dc8abbd721 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a38cfaebaa9a4ae8966419dc8abbd721" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 34017 } } peers { permanent_uuid: "2bfba7e0bfad4fe4a249d625797d1669" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 33487 } } peers { permanent_uuid: "0c70e98c59624aeda6118cfda612fa22" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 37599 } }
I20260619 10:17:10.659478 2445 raft_consensus.cc:374] T 11769cdadc70416c8459a6f4b6850276 P a38cfaebaa9a4ae8966419dc8abbd721 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 10:17:10.659507 2445 raft_consensus.cc:729] T 11769cdadc70416c8459a6f4b6850276 P a38cfaebaa9a4ae8966419dc8abbd721 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a38cfaebaa9a4ae8966419dc8abbd721, State: Initialized, Role: FOLLOWER
I20260619 10:17:10.659555 2445 consensus_queue.cc:260] T 11769cdadc70416c8459a6f4b6850276 P a38cfaebaa9a4ae8966419dc8abbd721 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a38cfaebaa9a4ae8966419dc8abbd721" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 34017 } } peers { permanent_uuid: "2bfba7e0bfad4fe4a249d625797d1669" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 33487 } } peers { permanent_uuid: "0c70e98c59624aeda6118cfda612fa22" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 37599 } }
I20260619 10:17:10.659653 2445 ts_tablet_manager.cc:1428] T 11769cdadc70416c8459a6f4b6850276 P a38cfaebaa9a4ae8966419dc8abbd721: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260619 10:17:10.659793 2444 tablet_replica.cc:439] T 11769cdadc70416c8459a6f4b6850276 P 2bfba7e0bfad4fe4a249d625797d1669: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a38cfaebaa9a4ae8966419dc8abbd721" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 34017 } } peers { permanent_uuid: "2bfba7e0bfad4fe4a249d625797d1669" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 33487 } } peers { permanent_uuid: "0c70e98c59624aeda6118cfda612fa22" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 37599 } } }
I20260619 10:17:10.659881 2444 tablet_replica.cc:442] T 11769cdadc70416c8459a6f4b6850276 P 2bfba7e0bfad4fe4a249d625797d1669: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 10:17:10.660063 2448 tablet_replica.cc:439] T 11769cdadc70416c8459a6f4b6850276 P a38cfaebaa9a4ae8966419dc8abbd721: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a38cfaebaa9a4ae8966419dc8abbd721" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 34017 } } peers { permanent_uuid: "2bfba7e0bfad4fe4a249d625797d1669" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 33487 } } peers { permanent_uuid: "0c70e98c59624aeda6118cfda612fa22" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 37599 } } }
I20260619 10:17:10.660147 2448 tablet_replica.cc:442] T 11769cdadc70416c8459a6f4b6850276 P a38cfaebaa9a4ae8966419dc8abbd721: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 10:17:10.874370 2460 raft_consensus.cc:482] T 11769cdadc70416c8459a6f4b6850276 P 0c70e98c59624aeda6118cfda612fa22 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 10:17:10.874495 2460 raft_consensus.cc:504] T 11769cdadc70416c8459a6f4b6850276 P 0c70e98c59624aeda6118cfda612fa22 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a38cfaebaa9a4ae8966419dc8abbd721" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 34017 } } peers { permanent_uuid: "2bfba7e0bfad4fe4a249d625797d1669" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 33487 } } peers { permanent_uuid: "0c70e98c59624aeda6118cfda612fa22" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 37599 } }
I20260619 10:17:10.874706 2460 leader_election.cc:288] T 11769cdadc70416c8459a6f4b6850276 P 0c70e98c59624aeda6118cfda612fa22 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a38cfaebaa9a4ae8966419dc8abbd721 (127.0.206.65:34017), 2bfba7e0bfad4fe4a249d625797d1669 (127.0.206.66:33487)
I20260619 10:17:10.874977 2094 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "11769cdadc70416c8459a6f4b6850276" candidate_uuid: "0c70e98c59624aeda6118cfda612fa22" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a38cfaebaa9a4ae8966419dc8abbd721" is_pre_election: true
I20260619 10:17:10.875123 2094 raft_consensus.cc:2454] T 11769cdadc70416c8459a6f4b6850276 P a38cfaebaa9a4ae8966419dc8abbd721 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0c70e98c59624aeda6118cfda612fa22 in term 0.
I20260619 10:17:10.875296 2234 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "11769cdadc70416c8459a6f4b6850276" candidate_uuid: "0c70e98c59624aeda6118cfda612fa22" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2bfba7e0bfad4fe4a249d625797d1669" is_pre_election: true
I20260619 10:17:10.875415 2234 raft_consensus.cc:2454] T 11769cdadc70416c8459a6f4b6850276 P 2bfba7e0bfad4fe4a249d625797d1669 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0c70e98c59624aeda6118cfda612fa22 in term 0.
I20260619 10:17:10.875581 2307 leader_election.cc:302] T 11769cdadc70416c8459a6f4b6850276 P 0c70e98c59624aeda6118cfda612fa22 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0c70e98c59624aeda6118cfda612fa22, 2bfba7e0bfad4fe4a249d625797d1669; no voters:
I20260619 10:17:10.875734 2460 raft_consensus.cc:2781] T 11769cdadc70416c8459a6f4b6850276 P 0c70e98c59624aeda6118cfda612fa22 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 10:17:10.875766 2460 raft_consensus.cc:482] T 11769cdadc70416c8459a6f4b6850276 P 0c70e98c59624aeda6118cfda612fa22 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 10:17:10.875792 2460 raft_consensus.cc:3037] T 11769cdadc70416c8459a6f4b6850276 P 0c70e98c59624aeda6118cfda612fa22 [term 0 FOLLOWER]: Advancing to term 1
I20260619 10:17:10.877287 2460 raft_consensus.cc:504] T 11769cdadc70416c8459a6f4b6850276 P 0c70e98c59624aeda6118cfda612fa22 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a38cfaebaa9a4ae8966419dc8abbd721" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 34017 } } peers { permanent_uuid: "2bfba7e0bfad4fe4a249d625797d1669" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 33487 } } peers { permanent_uuid: "0c70e98c59624aeda6118cfda612fa22" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 37599 } }
I20260619 10:17:10.877432 2460 leader_election.cc:288] T 11769cdadc70416c8459a6f4b6850276 P 0c70e98c59624aeda6118cfda612fa22 [CANDIDATE]: Term 1 election: Requested vote from peers a38cfaebaa9a4ae8966419dc8abbd721 (127.0.206.65:34017), 2bfba7e0bfad4fe4a249d625797d1669 (127.0.206.66:33487)
I20260619 10:17:10.877624 2234 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "11769cdadc70416c8459a6f4b6850276" candidate_uuid: "0c70e98c59624aeda6118cfda612fa22" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2bfba7e0bfad4fe4a249d625797d1669"
I20260619 10:17:10.877719 2234 raft_consensus.cc:3037] T 11769cdadc70416c8459a6f4b6850276 P 2bfba7e0bfad4fe4a249d625797d1669 [term 0 FOLLOWER]: Advancing to term 1
I20260619 10:17:10.878541 2234 raft_consensus.cc:2454] T 11769cdadc70416c8459a6f4b6850276 P 2bfba7e0bfad4fe4a249d625797d1669 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0c70e98c59624aeda6118cfda612fa22 in term 1.
I20260619 10:17:10.878770 2307 leader_election.cc:302] T 11769cdadc70416c8459a6f4b6850276 P 0c70e98c59624aeda6118cfda612fa22 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0c70e98c59624aeda6118cfda612fa22, 2bfba7e0bfad4fe4a249d625797d1669; no voters:
I20260619 10:17:10.878847 2460 raft_consensus.cc:2781] T 11769cdadc70416c8459a6f4b6850276 P 0c70e98c59624aeda6118cfda612fa22 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 10:17:10.878814 2094 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "11769cdadc70416c8459a6f4b6850276" candidate_uuid: "0c70e98c59624aeda6118cfda612fa22" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a38cfaebaa9a4ae8966419dc8abbd721"
I20260619 10:17:10.878885 2460 raft_consensus.cc:686] T 11769cdadc70416c8459a6f4b6850276 P 0c70e98c59624aeda6118cfda612fa22 [term 1 LEADER]: Becoming Leader. State: Replica: 0c70e98c59624aeda6118cfda612fa22, State: Running, Role: LEADER
I20260619 10:17:10.878922 2094 raft_consensus.cc:3037] T 11769cdadc70416c8459a6f4b6850276 P a38cfaebaa9a4ae8966419dc8abbd721 [term 0 FOLLOWER]: Advancing to term 1
I20260619 10:17:10.878954 2460 consensus_queue.cc:237] T 11769cdadc70416c8459a6f4b6850276 P 0c70e98c59624aeda6118cfda612fa22 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a38cfaebaa9a4ae8966419dc8abbd721" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 34017 } } peers { permanent_uuid: "2bfba7e0bfad4fe4a249d625797d1669" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 33487 } } peers { permanent_uuid: "0c70e98c59624aeda6118cfda612fa22" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 37599 } }
I20260619 10:17:10.879217 2460 tablet_replica.cc:439] T 11769cdadc70416c8459a6f4b6850276 P 0c70e98c59624aeda6118cfda612fa22: TxnStatusTablet state changed. Reason: New leader 0c70e98c59624aeda6118cfda612fa22. Latest consensus state: current_term: 1 leader_uuid: "0c70e98c59624aeda6118cfda612fa22" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a38cfaebaa9a4ae8966419dc8abbd721" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 34017 } } peers { permanent_uuid: "2bfba7e0bfad4fe4a249d625797d1669" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 33487 } } peers { permanent_uuid: "0c70e98c59624aeda6118cfda612fa22" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 37599 } } }
I20260619 10:17:10.879283 2460 tablet_replica.cc:442] T 11769cdadc70416c8459a6f4b6850276 P 0c70e98c59624aeda6118cfda612fa22: This TxnStatusTablet replica's current role is: LEADER
I20260619 10:17:10.879565 2482 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260619 10:17:10.879684 2482 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260619 10:17:10.879799 2094 raft_consensus.cc:2454] T 11769cdadc70416c8459a6f4b6850276 P a38cfaebaa9a4ae8966419dc8abbd721 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0c70e98c59624aeda6118cfda612fa22 in term 1.
I20260619 10:17:10.880219 1937 catalog_manager.cc:5495] T 11769cdadc70416c8459a6f4b6850276 P 0c70e98c59624aeda6118cfda612fa22 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0c70e98c59624aeda6118cfda612fa22 (127.0.206.67). New cstate: current_term: 1 leader_uuid: "0c70e98c59624aeda6118cfda612fa22" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a38cfaebaa9a4ae8966419dc8abbd721" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 34017 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2bfba7e0bfad4fe4a249d625797d1669" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 33487 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0c70e98c59624aeda6118cfda612fa22" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 37599 } health_report { overall_health: HEALTHY } } }
I20260619 10:17:10.889525 2438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:10.887171 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51584 (local address 127.0.206.67:37599)
0619 10:17:10.887210 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0619 10:17:10.887213 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 10:17:10.887260 (+ 47us) server_negotiation.cc:407] Connection header received
0619 10:17:10.887347 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:17:10.887348 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:17:10.887358 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:17:10.887411 (+ 53us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 10:17:10.887585 (+ 174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:10.888069 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:10.888821 (+ 752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:10.889028 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:10.889121 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:10.889311 (+ 190us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 10:17:10.889342 (+ 31us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 10:17:10.889380 (+ 38us) server_negotiation.cc:1035] Waiting for connection context
0619 10:17:10.889441 (+ 61us) server_negotiation.cc:299] Negotiation successful
0619 10:17:10.889459 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260619 10:17:10.889535 2477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:10.887137 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.67:37599 (local address 127.0.0.1:51584)
0619 10:17:10.887218 (+ 81us) negotiation.cc:107] Waiting for socket to connect
0619 10:17:10.887228 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 10:17:10.887280 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 10:17:10.887419 (+ 139us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 10:17:10.887419 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 10:17:10.887420 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0619 10:17:10.887477 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:10.887480 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:10.888151 (+ 671us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:10.888153 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:10.888722 (+ 569us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:10.888724 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:10.889122 (+ 398us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:10.889123 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:10.889206 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:10.889224 (+ 18us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0619 10:17:10.889390 (+ 166us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0619 10:17:10.889396 (+ 6us) client_negotiation.cc:769] Sending connection context
0619 10:17:10.889430 (+ 34us) client_negotiation.cc:240] Negotiation successful
0619 10:17:10.889474 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":46}
I20260619 10:17:10.890687 2094 raft_consensus.cc:1261] T 11769cdadc70416c8459a6f4b6850276 P a38cfaebaa9a4ae8966419dc8abbd721 [term 1 FOLLOWER]: Refusing update from remote peer 0c70e98c59624aeda6118cfda612fa22: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 10:17:10.890694 2234 raft_consensus.cc:1261] T 11769cdadc70416c8459a6f4b6850276 P 2bfba7e0bfad4fe4a249d625797d1669 [term 1 FOLLOWER]: Refusing update from remote peer 0c70e98c59624aeda6118cfda612fa22: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 10:17:10.890894 2450 consensus_queue.cc:1035] T 11769cdadc70416c8459a6f4b6850276 P 0c70e98c59624aeda6118cfda612fa22 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2bfba7e0bfad4fe4a249d625797d1669" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 33487 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 10:17:10.891041 2450 consensus_queue.cc:1035] T 11769cdadc70416c8459a6f4b6850276 P 0c70e98c59624aeda6118cfda612fa22 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a38cfaebaa9a4ae8966419dc8abbd721" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 34017 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 10:17:10.891417 2444 tablet_replica.cc:439] T 11769cdadc70416c8459a6f4b6850276 P 2bfba7e0bfad4fe4a249d625797d1669: TxnStatusTablet state changed. Reason: New leader 0c70e98c59624aeda6118cfda612fa22. Latest consensus state: current_term: 1 leader_uuid: "0c70e98c59624aeda6118cfda612fa22" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a38cfaebaa9a4ae8966419dc8abbd721" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 34017 } } peers { permanent_uuid: "2bfba7e0bfad4fe4a249d625797d1669" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 33487 } } peers { permanent_uuid: "0c70e98c59624aeda6118cfda612fa22" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 37599 } } }
I20260619 10:17:10.891520 2444 tablet_replica.cc:442] T 11769cdadc70416c8459a6f4b6850276 P 2bfba7e0bfad4fe4a249d625797d1669: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 10:17:10.891867 2448 tablet_replica.cc:439] T 11769cdadc70416c8459a6f4b6850276 P a38cfaebaa9a4ae8966419dc8abbd721: TxnStatusTablet state changed. Reason: New leader 0c70e98c59624aeda6118cfda612fa22. Latest consensus state: current_term: 1 leader_uuid: "0c70e98c59624aeda6118cfda612fa22" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a38cfaebaa9a4ae8966419dc8abbd721" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 34017 } } peers { permanent_uuid: "2bfba7e0bfad4fe4a249d625797d1669" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 33487 } } peers { permanent_uuid: "0c70e98c59624aeda6118cfda612fa22" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 37599 } } }
I20260619 10:17:10.891957 2448 tablet_replica.cc:442] T 11769cdadc70416c8459a6f4b6850276 P a38cfaebaa9a4ae8966419dc8abbd721: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 10:17:10.892087 2460 tablet_replica.cc:439] T 11769cdadc70416c8459a6f4b6850276 P 0c70e98c59624aeda6118cfda612fa22: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "0c70e98c59624aeda6118cfda612fa22" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a38cfaebaa9a4ae8966419dc8abbd721" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 34017 } } peers { permanent_uuid: "2bfba7e0bfad4fe4a249d625797d1669" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 33487 } } peers { permanent_uuid: "0c70e98c59624aeda6118cfda612fa22" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 37599 } } }
I20260619 10:17:10.892158 2460 tablet_replica.cc:442] T 11769cdadc70416c8459a6f4b6850276 P 0c70e98c59624aeda6118cfda612fa22: This TxnStatusTablet replica's current role is: LEADER
I20260619 10:17:10.892227 2453 mvcc.cc:205] Tried to move back new op lower bound from 7298515889726181376 to 7298515889680228352. Current Snapshot: MvccSnapshot[applied={T|T < 7298515889726181376 or (T in {7298515889726181376})}]
I20260619 10:17:10.892707 2444 tablet_replica.cc:439] T 11769cdadc70416c8459a6f4b6850276 P 2bfba7e0bfad4fe4a249d625797d1669: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "0c70e98c59624aeda6118cfda612fa22" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a38cfaebaa9a4ae8966419dc8abbd721" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 34017 } } peers { permanent_uuid: "2bfba7e0bfad4fe4a249d625797d1669" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 33487 } } peers { permanent_uuid: "0c70e98c59624aeda6118cfda612fa22" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 37599 } } }
I20260619 10:17:10.892794 2444 tablet_replica.cc:442] T 11769cdadc70416c8459a6f4b6850276 P 2bfba7e0bfad4fe4a249d625797d1669: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 10:17:10.894678 2460 tablet_replica.cc:439] T 11769cdadc70416c8459a6f4b6850276 P 0c70e98c59624aeda6118cfda612fa22: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "0c70e98c59624aeda6118cfda612fa22" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a38cfaebaa9a4ae8966419dc8abbd721" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 34017 } } peers { permanent_uuid: "2bfba7e0bfad4fe4a249d625797d1669" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 33487 } } peers { permanent_uuid: "0c70e98c59624aeda6118cfda612fa22" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 37599 } } }
I20260619 10:17:10.894748 2460 tablet_replica.cc:442] T 11769cdadc70416c8459a6f4b6850276 P 0c70e98c59624aeda6118cfda612fa22: This TxnStatusTablet replica's current role is: LEADER
I20260619 10:17:10.895166 2454 mvcc.cc:205] Tried to move back new op lower bound from 7298515889726181376 to 7298515889680228352. Current Snapshot: MvccSnapshot[applied={T|T < 7298515889726181376}]
I20260619 10:17:10.895254 2448 tablet_replica.cc:439] T 11769cdadc70416c8459a6f4b6850276 P a38cfaebaa9a4ae8966419dc8abbd721: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "0c70e98c59624aeda6118cfda612fa22" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a38cfaebaa9a4ae8966419dc8abbd721" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 34017 } } peers { permanent_uuid: "2bfba7e0bfad4fe4a249d625797d1669" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 33487 } } peers { permanent_uuid: "0c70e98c59624aeda6118cfda612fa22" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 37599 } } }
I20260619 10:17:10.895323 2448 tablet_replica.cc:442] T 11769cdadc70416c8459a6f4b6850276 P a38cfaebaa9a4ae8966419dc8abbd721: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 10:17:10.897137 2439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:10.894150 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49078 (local address 127.0.206.66:33487)
0619 10:17:10.894199 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0619 10:17:10.894201 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 10:17:10.894295 (+ 94us) server_negotiation.cc:407] Connection header received
0619 10:17:10.894330 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:17:10.894331 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:17:10.894351 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:17:10.894412 (+ 61us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 10:17:10.894582 (+ 170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:10.895267 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:10.896308 (+ 1041us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:10.896511 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:10.896602 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:10.896733 (+ 131us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 10:17:10.896840 (+ 107us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 10:17:10.896933 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0619 10:17:10.897042 (+ 109us) server_negotiation.cc:299] Negotiation successful
0619 10:17:10.897062 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260619 10:17:10.901495 2438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:10.899276 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.66:46085 (local address 127.0.206.67:37599)
0619 10:17:10.899312 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0619 10:17:10.899314 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 10:17:10.899452 (+ 138us) server_negotiation.cc:407] Connection header received
0619 10:17:10.899521 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:17:10.899522 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:17:10.899534 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:17:10.899584 (+ 50us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 10:17:10.899689 (+ 105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:10.900121 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:10.900831 (+ 710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:10.901025 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:10.901102 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:10.901273 (+ 171us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 10:17:10.901300 (+ 27us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 10:17:10.901365 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0619 10:17:10.901420 (+ 55us) server_negotiation.cc:299] Negotiation successful
0619 10:17:10.901436 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260619 10:17:10.901533 2489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:10.899223 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.67:37599 (local address 127.0.206.66:46085)
0619 10:17:10.899387 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0619 10:17:10.899396 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0619 10:17:10.899466 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 10:17:10.899587 (+ 121us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 10:17:10.899588 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 10:17:10.899589 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0619 10:17:10.899633 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:10.899634 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:10.900196 (+ 562us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:10.900197 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:10.900742 (+ 545us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:10.900744 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:10.901104 (+ 360us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:10.901105 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:10.901168 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:10.901186 (+ 18us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0619 10:17:10.901364 (+ 178us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0619 10:17:10.901366 (+ 2us) client_negotiation.cc:769] Sending connection context
0619 10:17:10.901415 (+ 49us) client_negotiation.cc:240] Negotiation successful
0619 10:17:10.901452 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":79,"threads_started":1}
I20260619 10:17:10.903633 2373 raft_consensus.cc:3037] T 76e4c8cc9da447b3a9b6e74ecbd162a5 P 0c70e98c59624aeda6118cfda612fa22 [term 0 FOLLOWER]: Advancing to term 1
I20260619 10:17:10.903709 2094 raft_consensus.cc:1261] T 76e4c8cc9da447b3a9b6e74ecbd162a5 P a38cfaebaa9a4ae8966419dc8abbd721 [term 1 FOLLOWER]: Refusing update from remote peer 2bfba7e0bfad4fe4a249d625797d1669: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 10:17:10.903889 2470 consensus_queue.cc:1035] T 76e4c8cc9da447b3a9b6e74ecbd162a5 P 2bfba7e0bfad4fe4a249d625797d1669 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a38cfaebaa9a4ae8966419dc8abbd721" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 34017 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 10:17:10.904443 2373 raft_consensus.cc:1261] T 76e4c8cc9da447b3a9b6e74ecbd162a5 P 0c70e98c59624aeda6118cfda612fa22 [term 1 FOLLOWER]: Refusing update from remote peer 2bfba7e0bfad4fe4a249d625797d1669: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 10:17:10.904644 2470 consensus_queue.cc:1035] T 76e4c8cc9da447b3a9b6e74ecbd162a5 P 2bfba7e0bfad4fe4a249d625797d1669 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0c70e98c59624aeda6118cfda612fa22" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 37599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 10:17:10.917320 2439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:10.914934 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.67:48387 (local address 127.0.206.66:33487)
0619 10:17:10.915022 (+ 88us) server_negotiation.cc:206] Beginning negotiation
0619 10:17:10.915024 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 10:17:10.915083 (+ 59us) server_negotiation.cc:407] Connection header received
0619 10:17:10.915150 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:17:10.915151 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:17:10.915160 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:17:10.915207 (+ 47us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 10:17:10.915294 (+ 87us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:10.915729 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:10.916584 (+ 855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:10.916862 (+ 278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:10.916905 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:10.917082 (+ 177us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 10:17:10.917112 (+ 30us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 10:17:10.917181 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0619 10:17:10.917240 (+ 59us) server_negotiation.cc:299] Negotiation successful
0619 10:17:10.917257 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260619 10:17:10.917356 2494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:10.914898 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.66:33487 (local address 127.0.206.67:48387)
0619 10:17:10.915035 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0619 10:17:10.915045 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 10:17:10.915098 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 10:17:10.915211 (+ 113us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 10:17:10.915211 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 10:17:10.915213 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0619 10:17:10.915250 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:10.915252 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:10.915816 (+ 564us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:10.915817 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:10.916495 (+ 678us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:10.916497 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:10.916949 (+ 452us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:10.916949 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:10.916988 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:10.916998 (+ 10us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0619 10:17:10.917181 (+ 183us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0619 10:17:10.917183 (+ 2us) client_negotiation.cc:769] Sending connection context
0619 10:17:10.917236 (+ 53us) client_negotiation.cc:240] Negotiation successful
0619 10:17:10.917279 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":95,"thread_start_us":75,"threads_started":1}
I20260619 10:17:10.925180 1936 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:38512:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 10:17:10.925275 1936 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:38512:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 10:17:10.925904 1936 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1f1f90ebcf834d238665f58d7af3e329: Sending DeleteTablet for 3 replicas of tablet 76e4c8cc9da447b3a9b6e74ecbd162a5
I20260619 10:17:10.926159 2354 tablet_service.cc:1514] Processing DeleteTablet for tablet 76e4c8cc9da447b3a9b6e74ecbd162a5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 10:17:10 UTC) from {username='kudu', principal='kudu/127.0.206.126@KRBTEST.COM'} at 127.0.0.1:51564
I20260619 10:17:10.926184 2214 tablet_service.cc:1514] Processing DeleteTablet for tablet 76e4c8cc9da447b3a9b6e74ecbd162a5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 10:17:10 UTC) from {username='kudu', principal='kudu/127.0.206.126@KRBTEST.COM'} at 127.0.0.1:49062
I20260619 10:17:10.926198 2072 tablet_service.cc:1514] Processing DeleteTablet for tablet 76e4c8cc9da447b3a9b6e74ecbd162a5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 10:17:10 UTC) from {username='kudu', principal='kudu/127.0.206.126@KRBTEST.COM'} at 127.0.0.1:52922
I20260619 10:17:10.926262 2464 tablet_replica.cc:330] T 76e4c8cc9da447b3a9b6e74ecbd162a5 P 0c70e98c59624aeda6118cfda612fa22: stopping tablet replica
I20260619 10:17:10.926306 2463 tablet_replica.cc:330] T 76e4c8cc9da447b3a9b6e74ecbd162a5 P a38cfaebaa9a4ae8966419dc8abbd721: stopping tablet replica
I20260619 10:17:10.926326 2464 raft_consensus.cc:2229] T 76e4c8cc9da447b3a9b6e74ecbd162a5 P 0c70e98c59624aeda6118cfda612fa22 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 10:17:10.926343 2463 raft_consensus.cc:2229] T 76e4c8cc9da447b3a9b6e74ecbd162a5 P a38cfaebaa9a4ae8966419dc8abbd721 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 10:17:10.926357 2464 raft_consensus.cc:2258] T 76e4c8cc9da447b3a9b6e74ecbd162a5 P 0c70e98c59624aeda6118cfda612fa22 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 10:17:10.926373 2463 raft_consensus.cc:2258] T 76e4c8cc9da447b3a9b6e74ecbd162a5 P a38cfaebaa9a4ae8966419dc8abbd721 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 10:17:10.926421 2465 tablet_replica.cc:330] T 76e4c8cc9da447b3a9b6e74ecbd162a5 P 2bfba7e0bfad4fe4a249d625797d1669: stopping tablet replica
I20260619 10:17:10.926486 2465 raft_consensus.cc:2229] T 76e4c8cc9da447b3a9b6e74ecbd162a5 P 2bfba7e0bfad4fe4a249d625797d1669 [term 1 LEADER]: Raft consensus shutting down.
I20260619 10:17:10.926554 2465 raft_consensus.cc:2258] T 76e4c8cc9da447b3a9b6e74ecbd162a5 P 2bfba7e0bfad4fe4a249d625797d1669 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 10:17:10.926963 2465 ts_tablet_manager.cc:1905] T 76e4c8cc9da447b3a9b6e74ecbd162a5 P 2bfba7e0bfad4fe4a249d625797d1669: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 10:17:10.927057 2463 ts_tablet_manager.cc:1905] T 76e4c8cc9da447b3a9b6e74ecbd162a5 P a38cfaebaa9a4ae8966419dc8abbd721: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 10:17:10.928638 2463 ts_tablet_manager.cc:1918] T 76e4c8cc9da447b3a9b6e74ecbd162a5 P a38cfaebaa9a4ae8966419dc8abbd721: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260619 10:17:10.928656 2465 ts_tablet_manager.cc:1918] T 76e4c8cc9da447b3a9b6e74ecbd162a5 P 2bfba7e0bfad4fe4a249d625797d1669: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260619 10:17:10.928678 2464 ts_tablet_manager.cc:1905] T 76e4c8cc9da447b3a9b6e74ecbd162a5 P 0c70e98c59624aeda6118cfda612fa22: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 10:17:10.928695 2463 log.cc:1196] T 76e4c8cc9da447b3a9b6e74ecbd162a5 P a38cfaebaa9a4ae8966419dc8abbd721: Deleting WAL directory at /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781864224577058-825-0/minicluster-data/ts-0/wal/wals/76e4c8cc9da447b3a9b6e74ecbd162a5
I20260619 10:17:10.928707 2465 log.cc:1196] T 76e4c8cc9da447b3a9b6e74ecbd162a5 P 2bfba7e0bfad4fe4a249d625797d1669: Deleting WAL directory at /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781864224577058-825-0/minicluster-data/ts-1/wal/wals/76e4c8cc9da447b3a9b6e74ecbd162a5
I20260619 10:17:10.928918 2463 ts_tablet_manager.cc:1939] T 76e4c8cc9da447b3a9b6e74ecbd162a5 P a38cfaebaa9a4ae8966419dc8abbd721: Deleting consensus metadata
I20260619 10:17:10.928923 2465 ts_tablet_manager.cc:1939] T 76e4c8cc9da447b3a9b6e74ecbd162a5 P 2bfba7e0bfad4fe4a249d625797d1669: Deleting consensus metadata
I20260619 10:17:10.929237 1921 catalog_manager.cc:4841] TS a38cfaebaa9a4ae8966419dc8abbd721 (127.0.206.65:34017): tablet 76e4c8cc9da447b3a9b6e74ecbd162a5 (table test-table [id=7c03db426678499ab0dfda29b4ea082c]) successfully deleted
I20260619 10:17:10.929219 1923 catalog_manager.cc:4841] TS 2bfba7e0bfad4fe4a249d625797d1669 (127.0.206.66:33487): tablet 76e4c8cc9da447b3a9b6e74ecbd162a5 (table test-table [id=7c03db426678499ab0dfda29b4ea082c]) successfully deleted
I20260619 10:17:10.930182 2464 ts_tablet_manager.cc:1918] T 76e4c8cc9da447b3a9b6e74ecbd162a5 P 0c70e98c59624aeda6118cfda612fa22: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260619 10:17:10.930229 2464 log.cc:1196] T 76e4c8cc9da447b3a9b6e74ecbd162a5 P 0c70e98c59624aeda6118cfda612fa22: Deleting WAL directory at /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781864224577058-825-0/minicluster-data/ts-2/wal/wals/76e4c8cc9da447b3a9b6e74ecbd162a5
I20260619 10:17:10.930435 2464 ts_tablet_manager.cc:1939] T 76e4c8cc9da447b3a9b6e74ecbd162a5 P 0c70e98c59624aeda6118cfda612fa22: Deleting consensus metadata
I20260619 10:17:10.930743 1922 catalog_manager.cc:4841] TS 0c70e98c59624aeda6118cfda612fa22 (127.0.206.67:37599): tablet 76e4c8cc9da447b3a9b6e74ecbd162a5 (table test-table [id=7c03db426678499ab0dfda29b4ea082c]) successfully deleted
Jun 19 10:17:10 dist-test-slave-6pt2 krb5kdc[1892](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864230, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.0.206.65@KRBTEST.COM
I20260619 10:17:10.939896 2437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:10.929536 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52924 (local address 127.0.206.65:34017)
0619 10:17:10.929573 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0619 10:17:10.929575 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 10:17:10.929845 (+ 270us) server_negotiation.cc:407] Connection header received
0619 10:17:10.929940 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:17:10.929941 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:17:10.929952 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:17:10.929982 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:17:10.930834 (+ 852us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:10.931615 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:10.932747 (+ 1132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:10.933067 (+ 320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:10.933144 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:10.936573 (+ 3429us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 10:17:10.936581 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 10:17:10.936581 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 10:17:10.936589 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 10:17:10.938554 (+ 1965us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:17:10.938972 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:17:10.938975 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:17:10.938976 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:17:10.939047 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:17:10.939256 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:17:10.939256 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:17:10.939257 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:17:10.939417 (+ 160us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 10:17:10.939556 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0619 10:17:10.939629 (+ 73us) server_negotiation.cc:299] Negotiation successful
0619 10:17:10.939708 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
W20260619 10:17:10.940096 2134 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:52924
I20260619 10:17:10.949761 2005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:10.940979 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38526 (local address 127.0.206.126:38679)
0619 10:17:10.941030 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0619 10:17:10.941032 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 10:17:10.942796 (+ 1764us) server_negotiation.cc:407] Connection header received
0619 10:17:10.942848 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:17:10.942849 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:17:10.942860 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:17:10.942922 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:17:10.943502 (+ 580us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:10.944271 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:10.945268 (+ 997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:10.945566 (+ 298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:10.945639 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:10.946958 (+ 1319us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 10:17:10.946960 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 10:17:10.946961 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 10:17:10.946975 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0619 10:17:10.948594 (+ 1619us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:17:10.948958 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:17:10.948959 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:17:10.948960 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:17:10.949016 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:17:10.949221 (+ 205us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:17:10.949222 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:17:10.949223 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:17:10.949349 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 10:17:10.949447 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0619 10:17:10.949596 (+ 149us) server_negotiation.cc:299] Negotiation successful
0619 10:17:10.949670 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
W20260619 10:17:10.949934 1936 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:38526
I20260619 10:17:10.950754 825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTU9OP6/build/release/bin/kudu with pid 2009
I20260619 10:17:10.957024 825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTU9OP6/build/release/bin/kudu with pid 2149
I20260619 10:17:10.962535 825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTU9OP6/build/release/bin/kudu with pid 2289
I20260619 10:17:10.968616 825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTU9OP6/build/release/bin/kudu with pid 1905
2026-06-19T10:17:10Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (2282 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781864224577058-825-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 10:17:11 dist-test-slave-6pt2 krb5kdc[2503](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 10:17:11 dist-test-slave-6pt2 krb5kdc[2503](info): set up 2 sockets
Jun 19 10:17:11 dist-test-slave-6pt2 krb5kdc[2503](info): commencing operation
krb5kdc: starting...
W20260619 10:17:11.986398 825 mini_kdc.cc:121] Time spent starting KDC: real 0.997s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781864224577058-825-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781864224577058-825-0/krb5kdc/test-user.keytab.
Jun 19 10:17:12 dist-test-slave-6pt2 krb5kdc[2503](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864232, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T10:17:12Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T10:17:12Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.206.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.206.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.206.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781864224577058-825-0/krb5kdc/kudu_127.0.206.126.keytab.
Entry for principal kudu/127.0.206.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781864224577058-825-0/krb5kdc/kudu_127.0.206.126.keytab.
I20260619 10:17:12.095052 825 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTU9OP6/build/release/bin/kudu
/tmp/dist-test-taskTU9OP6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781864224577058-825-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781864224577058-825-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781864224577058-825-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781864224577058-825-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.206.126:36165
--webserver_interface=127.0.206.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781864224577058-825-0/krb5kdc/kudu_127.0.206.126.keytab
--principal=kudu/127.0.206.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.206.84:46433
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.206.126:36165
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781864224577058-825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781864224577058-825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781864224577058-825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 10:17:12.173523 2516 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 10:17:12.173754 2516 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 10:17:12.173789 2516 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 10:17:12.174881 2516 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 10:17:12.174963 2516 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260619 10:17:12.174983 2516 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 10:17:12.175002 2516 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 10:17:12.175051 2516 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 10:17:12.175083 2516 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 10:17:12.176487 2516 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.206.84:46433
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781864224577058-825-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781864224577058-825-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.206.126:36165
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781864224577058-825-0/krb5kdc/kudu_127.0.206.126.keytab
--principal=kudu/127.0.206.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.206.126:36165
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781864224577058-825-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.206.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2516
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781864224577058-825-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 09:32:44 UTC on ccycloud.quasar-gbtldp.root.comops.site
build id 4455440
I20260619 10:17:12.176751 2516 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 10:17:12.176980 2516 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 10:17:12.181556 2524 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 10:17:12.181578 2522 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 10:17:12.181596 2521 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 10:17:12.181912 2516 server_base.cc:1034] running on GCE node
I20260619 10:17:12.182173 2516 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 10:17:12.182446 2516 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 10:17:12.183585 2516 hybrid_clock.cc:648] HybridClock initialized: now 1781864232183558 us; error 40 us; skew 500 ppm
Jun 19 10:17:12 dist-test-slave-6pt2 krb5kdc[2503](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864232, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 10:17:12.186226 2516 init.cc:377] Logged in from keytab as kudu/127.0.206.126@KRBTEST.COM (short username kudu)
I20260619 10:17:12.186658 2516 webserver.cc:460] Webserver started at http://127.0.206.126:33167/ using document root <none> and password file <none>
I20260619 10:17:12.186904 2516 fs_manager.cc:359] Metadata directory not provided
I20260619 10:17:12.186972 2516 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 10:17:12.187117 2516 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 10:17:12.188092 2516 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781864224577058-825-0/minicluster-data/master-0/data/instance:
uuid: "28ae00894a9046c291da0695c210585d"
format_stamp: "Formatted at 2026-06-19 10:17:12 on dist-test-slave-6pt2"
server_key: "214c73a2527d48f21da05e22d126e2f4"
server_key_iv: "844116b454aa42b73d184ac8b0cf7fd6"
server_key_version: "encryptionkey@0"
I20260619 10:17:12.188426 2516 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781864224577058-825-0/minicluster-data/master-0/wal/instance:
uuid: "28ae00894a9046c291da0695c210585d"
format_stamp: "Formatted at 2026-06-19 10:17:12 on dist-test-slave-6pt2"
server_key: "214c73a2527d48f21da05e22d126e2f4"
server_key_iv: "844116b454aa42b73d184ac8b0cf7fd6"
server_key_version: "encryptionkey@0"
I20260619 10:17:12.189729 2516 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 10:17:12.190407 2531 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 10:17:12.190655 2516 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 10:17:12.190769 2516 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781864224577058-825-0/minicluster-data/master-0/data,/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781864224577058-825-0/minicluster-data/master-0/wal
uuid: "28ae00894a9046c291da0695c210585d"
format_stamp: "Formatted at 2026-06-19 10:17:12 on dist-test-slave-6pt2"
server_key: "214c73a2527d48f21da05e22d126e2f4"
server_key_iv: "844116b454aa42b73d184ac8b0cf7fd6"
server_key_version: "encryptionkey@0"
I20260619 10:17:12.190853 2516 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781864224577058-825-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781864224577058-825-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781864224577058-825-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 10:17:12.198485 2516 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 10:17:12.200242 2516 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 10:17:12.200384 2516 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 10:17:12.204556 2516 rpc_server.cc:307] RPC server started. Bound to: 127.0.206.126:36165
I20260619 10:17:12.204620 2593 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.206.126:36165 every 8 connection(s)
I20260619 10:17:12.204914 2516 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781864224577058-825-0/minicluster-data/master-0/data/info.pb
I20260619 10:17:12.205533 2594 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 10:17:12.207628 2594 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 28ae00894a9046c291da0695c210585d: Bootstrap starting.
I20260619 10:17:12.208161 2594 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 28ae00894a9046c291da0695c210585d: Neither blocks nor log segments found. Creating new log.
I20260619 10:17:12.208469 2594 log.cc:824] T 00000000000000000000000000000000 P 28ae00894a9046c291da0695c210585d: Log is configured to *not* fsync() on all Append() calls
I20260619 10:17:12.209023 825 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTU9OP6/build/release/bin/kudu as pid 2516
I20260619 10:17:12.209117 825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781864224577058-825-0/minicluster-data/master-0/wal/instance
I20260619 10:17:12.209136 2594 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 28ae00894a9046c291da0695c210585d: No bootstrap required, opened a new log
I20260619 10:17:12.209277 825 external_mini_cluster.cc:1428] Setting key 0b665988785762d8378a7408fb0cc8de
I20260619 10:17:12.212001 2594 raft_consensus.cc:348] T 00000000000000000000000000000000 P 28ae00894a9046c291da0695c210585d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "28ae00894a9046c291da0695c210585d" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 36165 } }
I20260619 10:17:12.212154 2594 raft_consensus.cc:374] T 00000000000000000000000000000000 P 28ae00894a9046c291da0695c210585d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 10:17:12.212188 2594 raft_consensus.cc:729] T 00000000000000000000000000000000 P 28ae00894a9046c291da0695c210585d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 28ae00894a9046c291da0695c210585d, State: Initialized, Role: FOLLOWER
I20260619 10:17:12.212294 2594 consensus_queue.cc:260] T 00000000000000000000000000000000 P 28ae00894a9046c291da0695c210585d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "28ae00894a9046c291da0695c210585d" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 36165 } }
I20260619 10:17:12.212366 2594 raft_consensus.cc:388] T 00000000000000000000000000000000 P 28ae00894a9046c291da0695c210585d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 10:17:12.212392 2594 raft_consensus.cc:482] T 00000000000000000000000000000000 P 28ae00894a9046c291da0695c210585d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 10:17:12.212435 2594 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 28ae00894a9046c291da0695c210585d [term 0 FOLLOWER]: Advancing to term 1
I20260619 10:17:12.213037 2594 raft_consensus.cc:504] T 00000000000000000000000000000000 P 28ae00894a9046c291da0695c210585d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "28ae00894a9046c291da0695c210585d" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 36165 } }
I20260619 10:17:12.213115 2594 leader_election.cc:302] T 00000000000000000000000000000000 P 28ae00894a9046c291da0695c210585d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 28ae00894a9046c291da0695c210585d; no voters:
I20260619 10:17:12.213275 2594 leader_election.cc:288] T 00000000000000000000000000000000 P 28ae00894a9046c291da0695c210585d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 10:17:12.213296 2599 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 28ae00894a9046c291da0695c210585d [term 1 FOLLOWER]: Leader election won for term 1
I20260619 10:17:12.213394 2599 raft_consensus.cc:686] T 00000000000000000000000000000000 P 28ae00894a9046c291da0695c210585d [term 1 LEADER]: Becoming Leader. State: Replica: 28ae00894a9046c291da0695c210585d, State: Running, Role: LEADER
I20260619 10:17:12.213464 2599 consensus_queue.cc:237] T 00000000000000000000000000000000 P 28ae00894a9046c291da0695c210585d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "28ae00894a9046c291da0695c210585d" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 36165 } }
I20260619 10:17:12.213505 2594 sys_catalog.cc:564] T 00000000000000000000000000000000 P 28ae00894a9046c291da0695c210585d [sys.catalog]: configured and running, proceeding with master startup.
I20260619 10:17:12.213763 2599 sys_catalog.cc:455] T 00000000000000000000000000000000 P 28ae00894a9046c291da0695c210585d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "28ae00894a9046c291da0695c210585d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "28ae00894a9046c291da0695c210585d" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 36165 } } }
I20260619 10:17:12.213819 2599 sys_catalog.cc:458] T 00000000000000000000000000000000 P 28ae00894a9046c291da0695c210585d [sys.catalog]: This master's current role is: LEADER
I20260619 10:17:12.213936 2599 sys_catalog.cc:455] T 00000000000000000000000000000000 P 28ae00894a9046c291da0695c210585d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 28ae00894a9046c291da0695c210585d. Latest consensus state: current_term: 1 leader_uuid: "28ae00894a9046c291da0695c210585d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "28ae00894a9046c291da0695c210585d" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 36165 } } }
I20260619 10:17:12.213987 2599 sys_catalog.cc:458] T 00000000000000000000000000000000 P 28ae00894a9046c291da0695c210585d [sys.catalog]: This master's current role is: LEADER
W20260619 10:17:12.215332 2615 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 28ae00894a9046c291da0695c210585d: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 10:17:12.215386 2615 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 10:17:12.215438 2604 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 10:17:12.215591 2604 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Jun 19 10:17:12 dist-test-slave-6pt2 krb5kdc[2503](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864232, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.206.126@KRBTEST.COM
I20260619 10:17:12.217114 2604 catalog_manager.cc:1347] Generated new cluster ID: da83315b144f4af29e1ba902ddfb9018
I20260619 10:17:12.217160 2604 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 10:17:12.220322 2598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:12.209735 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36470 (local address 127.0.206.126:36165)
0619 10:17:12.209905 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0619 10:17:12.209914 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 10:17:12.209943 (+ 29us) server_negotiation.cc:407] Connection header received
0619 10:17:12.210503 (+ 560us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:17:12.210511 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:17:12.210566 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:17:12.210729 (+ 163us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:17:12.211296 (+ 567us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:12.212692 (+ 1396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:12.213517 (+ 825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:12.214089 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:12.214199 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:12.216974 (+ 2775us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 10:17:12.216977 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 10:17:12.216978 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 10:17:12.216988 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0619 10:17:12.218802 (+ 1814us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:17:12.219199 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:17:12.219202 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:17:12.219202 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:17:12.219294 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:17:12.219517 (+ 223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:17:12.219518 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:17:12.219518 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:17:12.219768 (+ 250us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 10:17:12.219913 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0619 10:17:12.220048 (+ 135us) server_negotiation.cc:299] Negotiation successful
0619 10:17:12.220152 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":122,"thread_start_us":75,"threads_started":1}
I20260619 10:17:12.221737 2604 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 10:17:12.222234 2604 catalog_manager.cc:1504] Loading token signing keys...
I20260619 10:17:12.226220 2604 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 28ae00894a9046c291da0695c210585d: Generated new TSK 0
I20260619 10:17:12.226380 2604 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.0.206.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.206.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.206.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781864224577058-825-0/krb5kdc/kudu_127.0.206.65.keytab.
Entry for principal kudu/127.0.206.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781864224577058-825-0/krb5kdc/kudu_127.0.206.65.keytab.
I20260619 10:17:12.251499 825 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTU9OP6/build/release/bin/kudu
/tmp/dist-test-taskTU9OP6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781864224577058-825-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781864224577058-825-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781864224577058-825-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781864224577058-825-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.206.65:0
--local_ip_for_outbound_sockets=127.0.206.65
--webserver_interface=127.0.206.65
--webserver_port=0
--tserver_master_addrs=127.0.206.126:36165
--keytab_file=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781864224577058-825-0/krb5kdc/kudu_127.0.206.65.keytab
--principal=kudu/127.0.206.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.206.84:46433
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781864224577058-825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781864224577058-825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781864224577058-825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 10:17:12.330360 2620 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 10:17:12.330557 2620 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 10:17:12.330582 2620 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 10:17:12.331831 2620 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 10:17:12.331879 2620 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 10:17:12.331902 2620 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260619 10:17:12.331914 2620 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260619 10:17:12.331931 2620 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.206.65
I20260619 10:17:12.333423 2620 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.206.84:46433
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781864224577058-825-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781864224577058-825-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781864224577058-825-0/krb5kdc/kudu_127.0.206.65.keytab
--principal=kudu/127.0.206.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.206.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781864224577058-825-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.206.65
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.0.206.126:36165
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2620
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.206.65
--log_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781864224577058-825-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 09:32:44 UTC on ccycloud.quasar-gbtldp.root.comops.site
build id 4455440
I20260619 10:17:12.333652 2620 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 10:17:12.333895 2620 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 10:17:12.339605 2625 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 10:17:12.339612 2626 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 10:17:12.339857 2620 server_base.cc:1034] running on GCE node
W20260619 10:17:12.339668 2628 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 10:17:12.340178 2620 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 10:17:12.340431 2620 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 10:17:12.341590 2620 hybrid_clock.cc:648] HybridClock initialized: now 1781864232341577 us; error 31 us; skew 500 ppm
Jun 19 10:17:12 dist-test-slave-6pt2 krb5kdc[2503](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864232, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 10:17:12.344033 2620 init.cc:377] Logged in from keytab as kudu/127.0.206.65@KRBTEST.COM (short username kudu)
I20260619 10:17:12.344462 2620 webserver.cc:460] Webserver started at http://127.0.206.65:37547/ using document root <none> and password file <none>
I20260619 10:17:12.344650 2620 fs_manager.cc:359] Metadata directory not provided
I20260619 10:17:12.344691 2620 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 10:17:12.344774 2620 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 10:17:12.345791 2620 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781864224577058-825-0/minicluster-data/ts-0/data/instance:
uuid: "b3a2757b35924642b12335f33832df9d"
format_stamp: "Formatted at 2026-06-19 10:17:12 on dist-test-slave-6pt2"
server_key: "0e6f3ac2aa9997ad55343dc8e7fff930"
server_key_iv: "f733580d0fc55075b2df5936abf36d6b"
server_key_version: "encryptionkey@0"
I20260619 10:17:12.346119 2620 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781864224577058-825-0/minicluster-data/ts-0/wal/instance:
uuid: "b3a2757b35924642b12335f33832df9d"
format_stamp: "Formatted at 2026-06-19 10:17:12 on dist-test-slave-6pt2"
server_key: "0e6f3ac2aa9997ad55343dc8e7fff930"
server_key_iv: "f733580d0fc55075b2df5936abf36d6b"
server_key_version: "encryptionkey@0"
I20260619 10:17:12.347424 2620 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260619 10:17:12.348101 2635 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 10:17:12.348286 2620 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260619 10:17:12.348349 2620 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781864224577058-825-0/minicluster-data/ts-0/data,/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781864224577058-825-0/minicluster-data/ts-0/wal
uuid: "b3a2757b35924642b12335f33832df9d"
format_stamp: "Formatted at 2026-06-19 10:17:12 on dist-test-slave-6pt2"
server_key: "0e6f3ac2aa9997ad55343dc8e7fff930"
server_key_iv: "f733580d0fc55075b2df5936abf36d6b"
server_key_version: "encryptionkey@0"
I20260619 10:17:12.348443 2620 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781864224577058-825-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781864224577058-825-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781864224577058-825-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 10:17:12.361863 2620 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 10:17:12.363795 2620 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 10:17:12.363957 2620 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 10:17:12.364421 2620 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 10:17:12.364467 2620 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 10:17:12.364492 2620 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 10:17:12.364563 2620 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Jun 19 10:17:12 dist-test-slave-6pt2 krb5kdc[2503](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864232, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.65@KRBTEST.COM for kudu/127.0.206.126@KRBTEST.COM
I20260619 10:17:12.371690 2620 rpc_server.cc:307] RPC server started. Bound to: 127.0.206.65:35095
I20260619 10:17:12.371800 2750 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.206.65:35095 every 8 connection(s)
I20260619 10:17:12.372116 2620 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781864224577058-825-0/minicluster-data/ts-0/data/info.pb
I20260619 10:17:12.373950 2598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:12.364989 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.65:60465 (local address 127.0.206.126:36165)
0619 10:17:12.365044 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0619 10:17:12.365046 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 10:17:12.365282 (+ 236us) server_negotiation.cc:407] Connection header received
0619 10:17:12.365851 (+ 569us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:17:12.365853 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:17:12.365866 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:17:12.365925 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:17:12.366881 (+ 956us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:12.367675 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:12.368671 (+ 996us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:12.368897 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:12.368983 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:12.371197 (+ 2214us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 10:17:12.371198 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 10:17:12.371199 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 10:17:12.371207 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 10:17:12.372867 (+ 1660us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:17:12.373202 (+ 335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:17:12.373203 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:17:12.373203 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:17:12.373251 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:17:12.373447 (+ 196us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:17:12.373448 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:17:12.373448 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:17:12.373543 (+ 95us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 10:17:12.373590 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0619 10:17:12.373807 (+ 217us) server_negotiation.cc:299] Negotiation successful
0619 10:17:12.373868 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260619 10:17:12.374145 2644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:12.364987 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:36165 (local address 127.0.206.65:60465)
0619 10:17:12.365169 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0619 10:17:12.365201 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0619 10:17:12.365758 (+ 557us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 10:17:12.366044 (+ 286us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 10:17:12.366049 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 10:17:12.366455 (+ 406us) client_negotiation.cc:189] Negotiated authn=SASL
0619 10:17:12.366777 (+ 322us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:12.366784 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:12.367766 (+ 982us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:12.367767 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:12.368563 (+ 796us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:12.368565 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:12.369006 (+ 441us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:12.369007 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:12.369179 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:12.369677 (+ 498us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 10:17:12.369687 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0619 10:17:12.371093 (+ 1406us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 10:17:12.372935 (+ 1842us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 10:17:12.372939 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 10:17:12.372940 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 10:17:12.373136 (+ 196us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 10:17:12.373306 (+ 170us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 10:17:12.373307 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 10:17:12.373307 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 10:17:12.373395 (+ 88us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 10:17:12.373594 (+ 199us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 10:17:12.373598 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 10:17:12.373711 (+ 113us) client_negotiation.cc:769] Sending connection context
0619 10:17:12.373790 (+ 79us) client_negotiation.cc:240] Negotiation successful
0619 10:17:12.373899 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":107,"threads_started":1}
I20260619 10:17:12.374684 2751 heartbeater.cc:347] Connected to a master server at 127.0.206.126:36165
I20260619 10:17:12.374764 2751 heartbeater.cc:464] Registering TS with master...
I20260619 10:17:12.374958 2751 heartbeater.cc:511] Master 127.0.206.126:36165 requested a full tablet report, sending...
I20260619 10:17:12.375524 2548 ts_manager.cc:194] Registered new tserver with Master: b3a2757b35924642b12335f33832df9d (127.0.206.65:35095)
I20260619 10:17:12.375684 825 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTU9OP6/build/release/bin/kudu as pid 2620
I20260619 10:17:12.375752 825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781864224577058-825-0/minicluster-data/ts-0/wal/instance
I20260619 10:17:12.375878 825 external_mini_cluster.cc:1428] Setting key 244510e880b3bd877f1e17e2cdd5d31a
I20260619 10:17:12.376425 2548 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.206.65@KRBTEST.COM'} at 127.0.206.65:60465
I20260619 10:17:12.382330 2757 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:12.376414 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:36165 (local address 127.0.206.65:55725)
0619 10:17:12.376572 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0619 10:17:12.376582 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 10:17:12.376640 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 10:17:12.377165 (+ 525us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 10:17:12.377166 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 10:17:12.377304 (+ 138us) client_negotiation.cc:189] Negotiated authn=SASL
0619 10:17:12.377344 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:12.377345 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:12.377961 (+ 616us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:12.377980 (+ 19us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:12.378487 (+ 507us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:12.378488 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:12.378885 (+ 397us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:12.378886 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:12.378965 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:12.379406 (+ 441us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 10:17:12.379412 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0619 10:17:12.379768 (+ 356us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 10:17:12.381317 (+ 1549us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 10:17:12.381317 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 10:17:12.381318 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 10:17:12.381484 (+ 166us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 10:17:12.381696 (+ 212us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 10:17:12.381697 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 10:17:12.381697 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 10:17:12.381752 (+ 55us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 10:17:12.382003 (+ 251us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 10:17:12.382004 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 10:17:12.382047 (+ 43us) client_negotiation.cc:769] Sending connection context
0619 10:17:12.382146 (+ 99us) client_negotiation.cc:240] Negotiation successful
0619 10:17:12.382241 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":68,"threads_started":1}
I20260619 10:17:12.382326 2598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:12.376458 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.65:55725 (local address 127.0.206.126:36165)
0619 10:17:12.376493 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0619 10:17:12.376494 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 10:17:12.376627 (+ 133us) server_negotiation.cc:407] Connection header received
0619 10:17:12.376730 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:17:12.376731 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:17:12.376740 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:17:12.376793 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:17:12.377432 (+ 639us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:12.377870 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:12.378571 (+ 701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:12.378798 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:12.378884 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:12.379865 (+ 981us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 10:17:12.379867 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 10:17:12.379867 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 10:17:12.379874 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 10:17:12.381220 (+ 1346us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:17:12.381568 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:17:12.381569 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:17:12.381570 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:17:12.381622 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:17:12.381824 (+ 202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:17:12.381825 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:17:12.381825 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:17:12.381935 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 10:17:12.382010 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0619 10:17:12.382178 (+ 168us) server_negotiation.cc:299] Negotiation successful
0619 10:17:12.382246 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
WARNING: no policy specified for kudu/127.0.206.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.206.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.206.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781864224577058-825-0/krb5kdc/kudu_127.0.206.66.keytab.
Entry for principal kudu/127.0.206.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781864224577058-825-0/krb5kdc/kudu_127.0.206.66.keytab.
I20260619 10:17:12.399570 825 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTU9OP6/build/release/bin/kudu
/tmp/dist-test-taskTU9OP6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781864224577058-825-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781864224577058-825-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781864224577058-825-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781864224577058-825-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.206.66:0
--local_ip_for_outbound_sockets=127.0.206.66
--webserver_interface=127.0.206.66
--webserver_port=0
--tserver_master_addrs=127.0.206.126:36165
--keytab_file=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781864224577058-825-0/krb5kdc/kudu_127.0.206.66.keytab
--principal=kudu/127.0.206.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.206.84:46433
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781864224577058-825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781864224577058-825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781864224577058-825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 10:17:12.478250 2760 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 10:17:12.478482 2760 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 10:17:12.478523 2760 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 10:17:12.479691 2760 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 10:17:12.479758 2760 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 10:17:12.479790 2760 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260619 10:17:12.479815 2760 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260619 10:17:12.479848 2760 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.206.66
I20260619 10:17:12.481256 2760 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.206.84:46433
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781864224577058-825-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781864224577058-825-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781864224577058-825-0/krb5kdc/kudu_127.0.206.66.keytab
--principal=kudu/127.0.206.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.206.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781864224577058-825-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.206.66
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.0.206.126:36165
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2760
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.206.66
--log_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781864224577058-825-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 09:32:44 UTC on ccycloud.quasar-gbtldp.root.comops.site
build id 4455440
I20260619 10:17:12.481487 2760 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 10:17:12.481724 2760 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 10:17:12.486462 2766 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 10:17:12.486493 2765 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 10:17:12.486572 2760 server_base.cc:1034] running on GCE node
W20260619 10:17:12.486462 2768 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 10:17:12.486912 2760 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 10:17:12.487151 2760 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 10:17:12.488309 2760 hybrid_clock.cc:648] HybridClock initialized: now 1781864232488291 us; error 30 us; skew 500 ppm
Jun 19 10:17:12 dist-test-slave-6pt2 krb5kdc[2503](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864232, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 10:17:12.490814 2760 init.cc:377] Logged in from keytab as kudu/127.0.206.66@KRBTEST.COM (short username kudu)
I20260619 10:17:12.491281 2760 webserver.cc:460] Webserver started at http://127.0.206.66:42087/ using document root <none> and password file <none>
I20260619 10:17:12.491503 2760 fs_manager.cc:359] Metadata directory not provided
I20260619 10:17:12.491569 2760 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 10:17:12.491719 2760 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 10:17:12.492729 2760 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781864224577058-825-0/minicluster-data/ts-1/data/instance:
uuid: "d869af37b51740718e3ea96f0972bc81"
format_stamp: "Formatted at 2026-06-19 10:17:12 on dist-test-slave-6pt2"
server_key: "d18e4209000f69ddd846ef500d5f420d"
server_key_iv: "b70fce962a25b89322bc440b0144db82"
server_key_version: "encryptionkey@0"
I20260619 10:17:12.493095 2760 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781864224577058-825-0/minicluster-data/ts-1/wal/instance:
uuid: "d869af37b51740718e3ea96f0972bc81"
format_stamp: "Formatted at 2026-06-19 10:17:12 on dist-test-slave-6pt2"
server_key: "d18e4209000f69ddd846ef500d5f420d"
server_key_iv: "b70fce962a25b89322bc440b0144db82"
server_key_version: "encryptionkey@0"
I20260619 10:17:12.494513 2760 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.002s
I20260619 10:17:12.495245 2775 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 10:17:12.495446 2760 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260619 10:17:12.495535 2760 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781864224577058-825-0/minicluster-data/ts-1/data,/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781864224577058-825-0/minicluster-data/ts-1/wal
uuid: "d869af37b51740718e3ea96f0972bc81"
format_stamp: "Formatted at 2026-06-19 10:17:12 on dist-test-slave-6pt2"
server_key: "d18e4209000f69ddd846ef500d5f420d"
server_key_iv: "b70fce962a25b89322bc440b0144db82"
server_key_version: "encryptionkey@0"
I20260619 10:17:12.495602 2760 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781864224577058-825-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781864224577058-825-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781864224577058-825-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 10:17:12.512466 2760 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 10:17:12.514755 2760 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 10:17:12.514922 2760 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 10:17:12.515513 2760 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 10:17:12.515561 2760 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 10:17:12.515595 2760 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 10:17:12.515622 2760 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Jun 19 10:17:12 dist-test-slave-6pt2 krb5kdc[2503](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864232, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.66@KRBTEST.COM for kudu/127.0.206.126@KRBTEST.COM
I20260619 10:17:12.522583 2760 rpc_server.cc:307] RPC server started. Bound to: 127.0.206.66:41427
I20260619 10:17:12.522926 2760 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781864224577058-825-0/minicluster-data/ts-1/data/info.pb
I20260619 10:17:12.522964 2890 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.206.66:41427 every 8 connection(s)
I20260619 10:17:12.523591 825 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTU9OP6/build/release/bin/kudu as pid 2760
I20260619 10:17:12.523690 825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781864224577058-825-0/minicluster-data/ts-1/wal/instance
I20260619 10:17:12.523850 825 external_mini_cluster.cc:1428] Setting key fba468232a2543f7f26cc57a27756827
I20260619 10:17:12.525115 2598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:12.515881 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.66:51827 (local address 127.0.206.126:36165)
0619 10:17:12.515952 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0619 10:17:12.515954 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 10:17:12.516182 (+ 228us) server_negotiation.cc:407] Connection header received
0619 10:17:12.516665 (+ 483us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:17:12.516666 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:17:12.516678 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:17:12.516721 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:17:12.517470 (+ 749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:12.518257 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:12.519387 (+ 1130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:12.519595 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:12.519679 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:12.522152 (+ 2473us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 10:17:12.522153 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 10:17:12.522154 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 10:17:12.522160 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 10:17:12.523714 (+ 1554us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:17:12.524169 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:17:12.524171 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:17:12.524171 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:17:12.524237 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:17:12.524485 (+ 248us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:17:12.524486 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:17:12.524487 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:17:12.524605 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 10:17:12.524681 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0619 10:17:12.524951 (+ 270us) server_negotiation.cc:299] Negotiation successful
0619 10:17:12.525028 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260619 10:17:12.525279 2784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:12.515858 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:36165 (local address 127.0.206.66:51827)
0619 10:17:12.516051 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0619 10:17:12.516089 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0619 10:17:12.516564 (+ 475us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 10:17:12.516793 (+ 229us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 10:17:12.516796 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 10:17:12.517109 (+ 313us) client_negotiation.cc:189] Negotiated authn=SASL
0619 10:17:12.517374 (+ 265us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:12.517395 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:12.518372 (+ 977us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:12.518373 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:12.519287 (+ 914us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:12.519289 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:12.519699 (+ 410us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:12.519700 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:12.519824 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:12.520305 (+ 481us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 10:17:12.520318 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0619 10:17:12.522049 (+ 1731us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 10:17:12.523802 (+ 1753us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 10:17:12.523807 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 10:17:12.523809 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 10:17:12.524091 (+ 282us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 10:17:12.524304 (+ 213us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 10:17:12.524305 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 10:17:12.524305 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 10:17:12.524416 (+ 111us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 10:17:12.524673 (+ 257us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 10:17:12.524679 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 10:17:12.524813 (+ 134us) client_negotiation.cc:769] Sending connection context
0619 10:17:12.524914 (+ 101us) client_negotiation.cc:240] Negotiation successful
0619 10:17:12.525063 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":131,"threads_started":1}
I20260619 10:17:12.526207 2891 heartbeater.cc:347] Connected to a master server at 127.0.206.126:36165
I20260619 10:17:12.526300 2891 heartbeater.cc:464] Registering TS with master...
I20260619 10:17:12.526472 2891 heartbeater.cc:511] Master 127.0.206.126:36165 requested a full tablet report, sending...
I20260619 10:17:12.526827 2548 ts_manager.cc:194] Registered new tserver with Master: d869af37b51740718e3ea96f0972bc81 (127.0.206.66:41427)
I20260619 10:17:12.527330 2548 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.206.66@KRBTEST.COM'} at 127.0.206.66:51827
WARNING: no policy specified for kudu/127.0.206.67@KRBTEST.COM; defaulting to no policy
I20260619 10:17:12.533196 2898 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:12.527215 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:36165 (local address 127.0.206.66:43145)
0619 10:17:12.527483 (+ 268us) negotiation.cc:107] Waiting for socket to connect
0619 10:17:12.527495 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 10:17:12.527554 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 10:17:12.527665 (+ 111us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 10:17:12.527666 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 10:17:12.527878 (+ 212us) client_negotiation.cc:189] Negotiated authn=SASL
0619 10:17:12.527950 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:12.527952 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:12.528600 (+ 648us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:12.528601 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:12.529526 (+ 925us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:12.529528 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:12.529853 (+ 325us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:12.529854 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:12.529939 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:12.530309 (+ 370us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 10:17:12.530315 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0619 10:17:12.530703 (+ 388us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 10:17:12.532283 (+ 1580us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 10:17:12.532284 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 10:17:12.532285 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 10:17:12.532494 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 10:17:12.532640 (+ 146us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 10:17:12.532641 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 10:17:12.532641 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 10:17:12.532690 (+ 49us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 10:17:12.532896 (+ 206us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 10:17:12.532897 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 10:17:12.532930 (+ 33us) client_negotiation.cc:769] Sending connection context
0619 10:17:12.533042 (+ 112us) client_negotiation.cc:240] Negotiation successful
0619 10:17:12.533129 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":243,"thread_start_us":81,"threads_started":1}
I20260619 10:17:12.533243 2598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:12.527255 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.66:43145 (local address 127.0.206.126:36165)
0619 10:17:12.527292 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0619 10:17:12.527293 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 10:17:12.527541 (+ 248us) server_negotiation.cc:407] Connection header received
0619 10:17:12.527598 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:17:12.527598 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:17:12.527609 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:17:12.527661 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:17:12.528036 (+ 375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:12.528501 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:12.529611 (+ 1110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:12.529780 (+ 169us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:12.529830 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:12.530780 (+ 950us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 10:17:12.530781 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 10:17:12.530782 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 10:17:12.530788 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 10:17:12.532210 (+ 1422us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:17:12.532556 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:17:12.532557 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:17:12.532557 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:17:12.532601 (+ 44us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:17:12.532726 (+ 125us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:17:12.532726 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:17:12.532726 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:17:12.532840 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 10:17:12.532897 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0619 10:17:12.533099 (+ 202us) server_negotiation.cc:299] Negotiation successful
0619 10:17:12.533175 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.206.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.206.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781864224577058-825-0/krb5kdc/kudu_127.0.206.67.keytab.
Entry for principal kudu/127.0.206.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781864224577058-825-0/krb5kdc/kudu_127.0.206.67.keytab.
I20260619 10:17:12.547272 825 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTU9OP6/build/release/bin/kudu
/tmp/dist-test-taskTU9OP6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781864224577058-825-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781864224577058-825-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781864224577058-825-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781864224577058-825-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.206.67:0
--local_ip_for_outbound_sockets=127.0.206.67
--webserver_interface=127.0.206.67
--webserver_port=0
--tserver_master_addrs=127.0.206.126:36165
--keytab_file=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781864224577058-825-0/krb5kdc/kudu_127.0.206.67.keytab
--principal=kudu/127.0.206.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.206.84:46433
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781864224577058-825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781864224577058-825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781864224577058-825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 10:17:12.631414 2900 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 10:17:12.631608 2900 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 10:17:12.631631 2900 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 10:17:12.632694 2900 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 10:17:12.632730 2900 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 10:17:12.632748 2900 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260619 10:17:12.632759 2900 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260619 10:17:12.632776 2900 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.206.67
I20260619 10:17:12.634142 2900 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.206.84:46433
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781864224577058-825-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781864224577058-825-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781864224577058-825-0/krb5kdc/kudu_127.0.206.67.keytab
--principal=kudu/127.0.206.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.206.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781864224577058-825-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.206.67
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.0.206.126:36165
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2900
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.206.67
--log_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781864224577058-825-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 09:32:44 UTC on ccycloud.quasar-gbtldp.root.comops.site
build id 4455440
I20260619 10:17:12.634325 2900 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 10:17:12.634500 2900 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 10:17:12.639313 2906 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 10:17:12.639334 2905 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 10:17:12.639505 2900 server_base.cc:1034] running on GCE node
W20260619 10:17:12.639366 2908 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 10:17:12.639786 2900 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 10:17:12.640048 2900 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 10:17:12.641204 2900 hybrid_clock.cc:648] HybridClock initialized: now 1781864232641199 us; error 35 us; skew 500 ppm
Jun 19 10:17:12 dist-test-slave-6pt2 krb5kdc[2503](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864232, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 10:17:12.643671 2900 init.cc:377] Logged in from keytab as kudu/127.0.206.67@KRBTEST.COM (short username kudu)
I20260619 10:17:12.644047 2900 webserver.cc:460] Webserver started at http://127.0.206.67:43371/ using document root <none> and password file <none>
I20260619 10:17:12.644240 2900 fs_manager.cc:359] Metadata directory not provided
I20260619 10:17:12.644280 2900 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 10:17:12.644376 2900 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 10:17:12.645357 2900 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781864224577058-825-0/minicluster-data/ts-2/data/instance:
uuid: "2b2d8effb39f4473b8bb107821ef1c81"
format_stamp: "Formatted at 2026-06-19 10:17:12 on dist-test-slave-6pt2"
server_key: "eb40858e56e02e1a856fa715c52b615a"
server_key_iv: "1be7b5930422a9443b429bf2b1b546a8"
server_key_version: "encryptionkey@0"
I20260619 10:17:12.645632 2900 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781864224577058-825-0/minicluster-data/ts-2/wal/instance:
uuid: "2b2d8effb39f4473b8bb107821ef1c81"
format_stamp: "Formatted at 2026-06-19 10:17:12 on dist-test-slave-6pt2"
server_key: "eb40858e56e02e1a856fa715c52b615a"
server_key_iv: "1be7b5930422a9443b429bf2b1b546a8"
server_key_version: "encryptionkey@0"
I20260619 10:17:12.646889 2900 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260619 10:17:12.647583 2915 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 10:17:12.647773 2900 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260619 10:17:12.647853 2900 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781864224577058-825-0/minicluster-data/ts-2/data,/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781864224577058-825-0/minicluster-data/ts-2/wal
uuid: "2b2d8effb39f4473b8bb107821ef1c81"
format_stamp: "Formatted at 2026-06-19 10:17:12 on dist-test-slave-6pt2"
server_key: "eb40858e56e02e1a856fa715c52b615a"
server_key_iv: "1be7b5930422a9443b429bf2b1b546a8"
server_key_version: "encryptionkey@0"
I20260619 10:17:12.647909 2900 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781864224577058-825-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781864224577058-825-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781864224577058-825-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 10:17:12.667932 2900 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 10:17:12.670243 2900 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 10:17:12.670353 2900 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 10:17:12.670853 2900 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 10:17:12.670893 2900 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 10:17:12.670918 2900 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 10:17:12.670970 2900 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 10:17:12.677621 2900 rpc_server.cc:307] RPC server started. Bound to: 127.0.206.67:35437
I20260619 10:17:12.677745 3030 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.206.67:35437 every 8 connection(s)
Jun 19 10:17:12 dist-test-slave-6pt2 krb5kdc[2503](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864232, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.67@KRBTEST.COM for kudu/127.0.206.126@KRBTEST.COM
I20260619 10:17:12.678041 2900 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781864224577058-825-0/minicluster-data/ts-2/data/info.pb
I20260619 10:17:12.680799 2598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:12.671371 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.67:39495 (local address 127.0.206.126:36165)
0619 10:17:12.671442 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0619 10:17:12.671445 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 10:17:12.671603 (+ 158us) server_negotiation.cc:407] Connection header received
0619 10:17:12.672116 (+ 513us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:17:12.672118 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:17:12.672132 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:17:12.672219 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:17:12.672902 (+ 683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:12.673726 (+ 824us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:12.675389 (+ 1663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:12.675769 (+ 380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:12.675855 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:12.678543 (+ 2688us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 10:17:12.678544 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 10:17:12.678545 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 10:17:12.678553 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 10:17:12.679769 (+ 1216us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:17:12.680119 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:17:12.680119 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:17:12.680120 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:17:12.680166 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:17:12.680337 (+ 171us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:17:12.680338 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:17:12.680338 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:17:12.680434 (+ 96us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 10:17:12.680482 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0619 10:17:12.680682 (+ 200us) server_negotiation.cc:299] Negotiation successful
0619 10:17:12.680737 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260619 10:17:12.680968 2924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:12.671323 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:36165 (local address 127.0.206.67:39495)
0619 10:17:12.671479 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0619 10:17:12.671519 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0619 10:17:12.672016 (+ 497us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 10:17:12.672319 (+ 303us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 10:17:12.672323 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 10:17:12.672612 (+ 289us) client_negotiation.cc:189] Negotiated authn=SASL
0619 10:17:12.672788 (+ 176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:12.672793 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:12.673820 (+ 1027us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:12.673821 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:12.675286 (+ 1465us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:12.675289 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:12.675878 (+ 589us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:12.675879 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:12.676032 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:12.676564 (+ 532us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 10:17:12.676574 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0619 10:17:12.678440 (+ 1866us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 10:17:12.679836 (+ 1396us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 10:17:12.679840 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 10:17:12.679842 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 10:17:12.680054 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 10:17:12.680207 (+ 153us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 10:17:12.680207 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 10:17:12.680207 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 10:17:12.680283 (+ 76us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 10:17:12.680488 (+ 205us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 10:17:12.680492 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 10:17:12.680594 (+ 102us) client_negotiation.cc:769] Sending connection context
0619 10:17:12.680671 (+ 77us) client_negotiation.cc:240] Negotiation successful
0619 10:17:12.680779 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":90,"threads_started":1}
I20260619 10:17:12.681459 3031 heartbeater.cc:347] Connected to a master server at 127.0.206.126:36165
I20260619 10:17:12.681535 3031 heartbeater.cc:464] Registering TS with master...
I20260619 10:17:12.681697 3031 heartbeater.cc:511] Master 127.0.206.126:36165 requested a full tablet report, sending...
I20260619 10:17:12.682044 2547 ts_manager.cc:194] Registered new tserver with Master: 2b2d8effb39f4473b8bb107821ef1c81 (127.0.206.67:35437)
I20260619 10:17:12.682503 2547 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.206.67@KRBTEST.COM'} at 127.0.206.67:39495
I20260619 10:17:12.683873 825 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTU9OP6/build/release/bin/kudu as pid 2900
I20260619 10:17:12.683979 825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781864224577058-825-0/minicluster-data/ts-2/wal/instance
I20260619 10:17:12.684154 825 external_mini_cluster.cc:1428] Setting key c16aafa47cca0430af458d3fef014b70
I20260619 10:17:12.685019 825 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 10:17:12.688045 3037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:12.682422 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:36165 (local address 127.0.206.67:56503)
0619 10:17:12.682549 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0619 10:17:12.682558 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0619 10:17:12.682642 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 10:17:12.682766 (+ 124us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 10:17:12.682766 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 10:17:12.682957 (+ 191us) client_negotiation.cc:189] Negotiated authn=SASL
0619 10:17:12.683020 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:12.683022 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:12.683734 (+ 712us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:12.683735 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:12.684203 (+ 468us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:12.684204 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:12.684709 (+ 505us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:12.684710 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:12.684778 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:12.685233 (+ 455us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 10:17:12.685238 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 10:17:12.685585 (+ 347us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 10:17:12.687155 (+ 1570us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 10:17:12.687155 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 10:17:12.687155 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 10:17:12.687322 (+ 167us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 10:17:12.687527 (+ 205us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 10:17:12.687528 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 10:17:12.687528 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 10:17:12.687574 (+ 46us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 10:17:12.687841 (+ 267us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 10:17:12.687841 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 10:17:12.687873 (+ 32us) client_negotiation.cc:769] Sending connection context
0619 10:17:12.687922 (+ 49us) client_negotiation.cc:240] Negotiation successful
0619 10:17:12.687988 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":76,"threads_started":1}
I20260619 10:17:12.688148 2598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:12.682487 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.67:56503 (local address 127.0.206.126:36165)
0619 10:17:12.682643 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0619 10:17:12.682644 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 10:17:12.682653 (+ 9us) server_negotiation.cc:407] Connection header received
0619 10:17:12.682694 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:17:12.682695 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:17:12.682704 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:17:12.682761 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:17:12.683112 (+ 351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:12.683641 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:12.684285 (+ 644us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:12.684561 (+ 276us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:12.684660 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:12.685687 (+ 1027us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 10:17:12.685688 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 10:17:12.685689 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 10:17:12.685694 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0619 10:17:12.687065 (+ 1371us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:17:12.687400 (+ 335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:17:12.687401 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:17:12.687401 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:17:12.687462 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:17:12.687625 (+ 163us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:17:12.687626 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:17:12.687626 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:17:12.687762 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 10:17:12.687841 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0619 10:17:12.687957 (+ 116us) server_negotiation.cc:299] Negotiation successful
0619 10:17:12.688031 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":140}
Jun 19 10:17:12 dist-test-slave-6pt2 krb5kdc[2503](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864232, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Jun 19 10:17:12 dist-test-slave-6pt2 krb5kdc[2503](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864232, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.206.126@KRBTEST.COM
I20260619 10:17:12.707501 2598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:12.701091 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36476 (local address 127.0.206.126:36165)
0619 10:17:12.701143 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0619 10:17:12.701146 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 10:17:12.701156 (+ 10us) server_negotiation.cc:407] Connection header received
0619 10:17:12.701180 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:17:12.701181 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:17:12.701191 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:17:12.701259 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:17:12.701745 (+ 486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:12.702236 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:12.702990 (+ 754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:12.703159 (+ 169us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:12.703211 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:12.705300 (+ 2089us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 10:17:12.705309 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 10:17:12.705309 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 10:17:12.705316 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 10:17:12.706675 (+ 1359us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:17:12.706951 (+ 276us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:17:12.706951 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:17:12.706952 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:17:12.706995 (+ 43us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:17:12.707138 (+ 143us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:17:12.707139 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:17:12.707139 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:17:12.707219 (+ 80us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 10:17:12.707281 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0619 10:17:12.707360 (+ 79us) server_negotiation.cc:299] Negotiation successful
0619 10:17:12.707409 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260619 10:17:12.709086 2547 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36476:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 10:17:12.710502 2547 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 10:17:12.717020 3049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:12.712576 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.67:35437 (local address 127.0.0.1:48628)
0619 10:17:12.712843 (+ 267us) negotiation.cc:107] Waiting for socket to connect
0619 10:17:12.712853 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 10:17:12.712895 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 10:17:12.713425 (+ 530us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 10:17:12.713426 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 10:17:12.713428 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 10:17:12.713478 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:12.713480 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:12.714513 (+ 1033us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:12.714514 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:12.715850 (+ 1336us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:12.715852 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:12.716758 (+ 906us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:12.716759 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:12.716818 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:12.716822 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 10:17:12.716851 (+ 29us) client_negotiation.cc:240] Negotiation successful
0619 10:17:12.716909 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":257,"mutex_wait_us":35,"thread_start_us":52,"threads_started":1}
I20260619 10:17:12.717031 3047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:12.712668 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57326 (local address 127.0.206.65:35095)
0619 10:17:12.712837 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0619 10:17:12.712843 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 10:17:12.712855 (+ 12us) server_negotiation.cc:407] Connection header received
0619 10:17:12.712983 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:17:12.712987 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:17:12.713001 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:17:12.713154 (+ 153us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 10:17:12.713288 (+ 134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:12.714276 (+ 988us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:12.716032 (+ 1756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:12.716595 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:12.716684 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:12.716743 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0619 10:17:12.716902 (+ 159us) server_negotiation.cc:299] Negotiation successful
0619 10:17:12.716940 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":139,"thread_start_us":66,"threads_started":1}
I20260619 10:17:12.717155 3045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:12.712576 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.65:35095 (local address 127.0.0.1:57326)
0619 10:17:12.712739 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0619 10:17:12.712777 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0619 10:17:12.712851 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 10:17:12.713160 (+ 309us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 10:17:12.713163 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 10:17:12.713165 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 10:17:12.713211 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:12.713212 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:12.714352 (+ 1140us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:12.714353 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:12.715953 (+ 1600us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:12.715960 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:12.716695 (+ 735us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:12.716696 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:12.716777 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:12.716789 (+ 12us) client_negotiation.cc:769] Sending connection context
0619 10:17:12.716885 (+ 96us) client_negotiation.cc:240] Negotiation successful
0619 10:17:12.716936 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141,"mutex_wait_us":37,"thread_start_us":56,"threads_started":1}
I20260619 10:17:12.717207 3048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:12.712654 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48628 (local address 127.0.206.67:35437)
0619 10:17:12.713273 (+ 619us) server_negotiation.cc:206] Beginning negotiation
0619 10:17:12.713276 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 10:17:12.713285 (+ 9us) server_negotiation.cc:407] Connection header received
0619 10:17:12.713316 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:17:12.713318 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:17:12.713356 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:17:12.713425 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 10:17:12.713559 (+ 134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:12.714370 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:12.715938 (+ 1568us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:12.716670 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:12.716984 (+ 314us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:12.717044 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0619 10:17:12.717069 (+ 25us) server_negotiation.cc:299] Negotiation successful
0619 10:17:12.717110 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":585,"thread_start_us":87,"threads_started":1}
I20260619 10:17:12.717936 3046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:12.712767 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.66:41427 (local address 127.0.0.1:51598)
0619 10:17:12.712917 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0619 10:17:12.712927 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 10:17:12.712979 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 10:17:12.714417 (+ 1438us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 10:17:12.714418 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 10:17:12.714420 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 10:17:12.714457 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:12.714458 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:12.715504 (+ 1046us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:12.715505 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:12.716603 (+ 1098us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:12.716605 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:12.717635 (+ 1030us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:12.717636 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:12.717761 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:12.717766 (+ 5us) client_negotiation.cc:769] Sending connection context
0619 10:17:12.717814 (+ 48us) client_negotiation.cc:240] Negotiation successful
0619 10:17:12.717852 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":51,"threads_started":1}
I20260619 10:17:12.717871 2965 tablet_service.cc:1467] Processing CreateTablet for tablet 7535de8b335d4947bcc11e3c6aa0d418 (DEFAULT_TABLE table=test-table [id=c72ae3e8e59c4e229a1bef42057fbb77]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 10:17:12.718189 3050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:12.712943 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51598 (local address 127.0.206.66:41427)
0619 10:17:12.713126 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0619 10:17:12.713130 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 10:17:12.713140 (+ 10us) server_negotiation.cc:407] Connection header received
0619 10:17:12.713176 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:17:12.713178 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:17:12.713192 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:17:12.713273 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 10:17:12.714519 (+ 1246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:12.715391 (+ 872us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:12.716704 (+ 1313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:12.717537 (+ 833us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:12.717630 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:12.717684 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0619 10:17:12.718035 (+ 351us) server_negotiation.cc:299] Negotiation successful
0619 10:17:12.718081 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":146,"thread_start_us":92,"threads_started":1}
I20260619 10:17:12.718091 2685 tablet_service.cc:1467] Processing CreateTablet for tablet 7535de8b335d4947bcc11e3c6aa0d418 (DEFAULT_TABLE table=test-table [id=c72ae3e8e59c4e229a1bef42057fbb77]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 10:17:12.718228 2965 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7535de8b335d4947bcc11e3c6aa0d418. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 10:17:12.718389 2685 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7535de8b335d4947bcc11e3c6aa0d418. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 10:17:12.721515 3051 tablet_bootstrap.cc:492] T 7535de8b335d4947bcc11e3c6aa0d418 P 2b2d8effb39f4473b8bb107821ef1c81: Bootstrap starting.
I20260619 10:17:12.721844 3052 tablet_bootstrap.cc:492] T 7535de8b335d4947bcc11e3c6aa0d418 P b3a2757b35924642b12335f33832df9d: Bootstrap starting.
I20260619 10:17:12.722378 3051 tablet_bootstrap.cc:654] T 7535de8b335d4947bcc11e3c6aa0d418 P 2b2d8effb39f4473b8bb107821ef1c81: Neither blocks nor log segments found. Creating new log.
I20260619 10:17:12.722318 2825 tablet_service.cc:1467] Processing CreateTablet for tablet 7535de8b335d4947bcc11e3c6aa0d418 (DEFAULT_TABLE table=test-table [id=c72ae3e8e59c4e229a1bef42057fbb77]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 10:17:12.722487 3052 tablet_bootstrap.cc:654] T 7535de8b335d4947bcc11e3c6aa0d418 P b3a2757b35924642b12335f33832df9d: Neither blocks nor log segments found. Creating new log.
I20260619 10:17:12.722707 2825 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7535de8b335d4947bcc11e3c6aa0d418. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 10:17:12.722792 3051 log.cc:824] T 7535de8b335d4947bcc11e3c6aa0d418 P 2b2d8effb39f4473b8bb107821ef1c81: Log is configured to *not* fsync() on all Append() calls
I20260619 10:17:12.722834 3052 log.cc:824] T 7535de8b335d4947bcc11e3c6aa0d418 P b3a2757b35924642b12335f33832df9d: Log is configured to *not* fsync() on all Append() calls
I20260619 10:17:12.723461 3051 tablet_bootstrap.cc:492] T 7535de8b335d4947bcc11e3c6aa0d418 P 2b2d8effb39f4473b8bb107821ef1c81: No bootstrap required, opened a new log
I20260619 10:17:12.723537 3051 ts_tablet_manager.cc:1397] T 7535de8b335d4947bcc11e3c6aa0d418 P 2b2d8effb39f4473b8bb107821ef1c81: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260619 10:17:12.725032 3051 raft_consensus.cc:348] T 7535de8b335d4947bcc11e3c6aa0d418 P 2b2d8effb39f4473b8bb107821ef1c81 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3a2757b35924642b12335f33832df9d" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 35095 } } peers { permanent_uuid: "2b2d8effb39f4473b8bb107821ef1c81" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 35437 } } peers { permanent_uuid: "d869af37b51740718e3ea96f0972bc81" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 41427 } }
I20260619 10:17:12.725158 3051 raft_consensus.cc:374] T 7535de8b335d4947bcc11e3c6aa0d418 P 2b2d8effb39f4473b8bb107821ef1c81 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 10:17:12.725179 3051 raft_consensus.cc:729] T 7535de8b335d4947bcc11e3c6aa0d418 P 2b2d8effb39f4473b8bb107821ef1c81 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2b2d8effb39f4473b8bb107821ef1c81, State: Initialized, Role: FOLLOWER
I20260619 10:17:12.725298 3051 consensus_queue.cc:260] T 7535de8b335d4947bcc11e3c6aa0d418 P 2b2d8effb39f4473b8bb107821ef1c81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3a2757b35924642b12335f33832df9d" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 35095 } } peers { permanent_uuid: "2b2d8effb39f4473b8bb107821ef1c81" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 35437 } } peers { permanent_uuid: "d869af37b51740718e3ea96f0972bc81" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 41427 } }
I20260619 10:17:12.725566 3051 ts_tablet_manager.cc:1428] T 7535de8b335d4947bcc11e3c6aa0d418 P 2b2d8effb39f4473b8bb107821ef1c81: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260619 10:17:12.725585 3055 tablet_bootstrap.cc:492] T 7535de8b335d4947bcc11e3c6aa0d418 P d869af37b51740718e3ea96f0972bc81: Bootstrap starting.
I20260619 10:17:12.725605 3031 heartbeater.cc:503] Master 127.0.206.126:36165 was elected leader, sending a full tablet report...
I20260619 10:17:12.726186 3052 tablet_bootstrap.cc:492] T 7535de8b335d4947bcc11e3c6aa0d418 P b3a2757b35924642b12335f33832df9d: No bootstrap required, opened a new log
I20260619 10:17:12.726207 3055 tablet_bootstrap.cc:654] T 7535de8b335d4947bcc11e3c6aa0d418 P d869af37b51740718e3ea96f0972bc81: Neither blocks nor log segments found. Creating new log.
I20260619 10:17:12.726245 3052 ts_tablet_manager.cc:1397] T 7535de8b335d4947bcc11e3c6aa0d418 P b3a2757b35924642b12335f33832df9d: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20260619 10:17:12.726498 3055 log.cc:824] T 7535de8b335d4947bcc11e3c6aa0d418 P d869af37b51740718e3ea96f0972bc81: Log is configured to *not* fsync() on all Append() calls
I20260619 10:17:12.727185 3055 tablet_bootstrap.cc:492] T 7535de8b335d4947bcc11e3c6aa0d418 P d869af37b51740718e3ea96f0972bc81: No bootstrap required, opened a new log
I20260619 10:17:12.727267 3055 ts_tablet_manager.cc:1397] T 7535de8b335d4947bcc11e3c6aa0d418 P d869af37b51740718e3ea96f0972bc81: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260619 10:17:12.727885 3052 raft_consensus.cc:348] T 7535de8b335d4947bcc11e3c6aa0d418 P b3a2757b35924642b12335f33832df9d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3a2757b35924642b12335f33832df9d" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 35095 } } peers { permanent_uuid: "2b2d8effb39f4473b8bb107821ef1c81" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 35437 } } peers { permanent_uuid: "d869af37b51740718e3ea96f0972bc81" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 41427 } }
I20260619 10:17:12.728004 3052 raft_consensus.cc:374] T 7535de8b335d4947bcc11e3c6aa0d418 P b3a2757b35924642b12335f33832df9d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 10:17:12.728026 3052 raft_consensus.cc:729] T 7535de8b335d4947bcc11e3c6aa0d418 P b3a2757b35924642b12335f33832df9d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b3a2757b35924642b12335f33832df9d, State: Initialized, Role: FOLLOWER
I20260619 10:17:12.728157 3052 consensus_queue.cc:260] T 7535de8b335d4947bcc11e3c6aa0d418 P b3a2757b35924642b12335f33832df9d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3a2757b35924642b12335f33832df9d" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 35095 } } peers { permanent_uuid: "2b2d8effb39f4473b8bb107821ef1c81" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 35437 } } peers { permanent_uuid: "d869af37b51740718e3ea96f0972bc81" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 41427 } }
I20260619 10:17:12.728467 3052 ts_tablet_manager.cc:1428] T 7535de8b335d4947bcc11e3c6aa0d418 P b3a2757b35924642b12335f33832df9d: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260619 10:17:12.728598 2751 heartbeater.cc:503] Master 127.0.206.126:36165 was elected leader, sending a full tablet report...
I20260619 10:17:12.728910 3055 raft_consensus.cc:348] T 7535de8b335d4947bcc11e3c6aa0d418 P d869af37b51740718e3ea96f0972bc81 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3a2757b35924642b12335f33832df9d" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 35095 } } peers { permanent_uuid: "2b2d8effb39f4473b8bb107821ef1c81" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 35437 } } peers { permanent_uuid: "d869af37b51740718e3ea96f0972bc81" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 41427 } }
I20260619 10:17:12.729018 3055 raft_consensus.cc:374] T 7535de8b335d4947bcc11e3c6aa0d418 P d869af37b51740718e3ea96f0972bc81 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 10:17:12.729068 3055 raft_consensus.cc:729] T 7535de8b335d4947bcc11e3c6aa0d418 P d869af37b51740718e3ea96f0972bc81 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d869af37b51740718e3ea96f0972bc81, State: Initialized, Role: FOLLOWER
I20260619 10:17:12.729179 3055 consensus_queue.cc:260] T 7535de8b335d4947bcc11e3c6aa0d418 P d869af37b51740718e3ea96f0972bc81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3a2757b35924642b12335f33832df9d" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 35095 } } peers { permanent_uuid: "2b2d8effb39f4473b8bb107821ef1c81" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 35437 } } peers { permanent_uuid: "d869af37b51740718e3ea96f0972bc81" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 41427 } }
I20260619 10:17:12.729429 3055 ts_tablet_manager.cc:1428] T 7535de8b335d4947bcc11e3c6aa0d418 P d869af37b51740718e3ea96f0972bc81: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260619 10:17:12.729558 2891 heartbeater.cc:503] Master 127.0.206.126:36165 was elected leader, sending a full tablet report...
I20260619 10:17:12.730388 3058 raft_consensus.cc:482] T 7535de8b335d4947bcc11e3c6aa0d418 P b3a2757b35924642b12335f33832df9d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 10:17:12.730468 3058 raft_consensus.cc:504] T 7535de8b335d4947bcc11e3c6aa0d418 P b3a2757b35924642b12335f33832df9d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3a2757b35924642b12335f33832df9d" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 35095 } } peers { permanent_uuid: "2b2d8effb39f4473b8bb107821ef1c81" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 35437 } } peers { permanent_uuid: "d869af37b51740718e3ea96f0972bc81" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 41427 } }
I20260619 10:17:12.730798 3058 leader_election.cc:288] T 7535de8b335d4947bcc11e3c6aa0d418 P b3a2757b35924642b12335f33832df9d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2b2d8effb39f4473b8bb107821ef1c81 (127.0.206.67:35437), d869af37b51740718e3ea96f0972bc81 (127.0.206.66:41427)
I20260619 10:17:12.734261 2644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:12.730910 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.67:35437 (local address 127.0.206.65:38751)
0619 10:17:12.730949 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0619 10:17:12.730960 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 10:17:12.731002 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 10:17:12.731162 (+ 160us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 10:17:12.731173 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 10:17:12.731175 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 10:17:12.731239 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:12.731240 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:12.732129 (+ 889us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:12.732130 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:12.733315 (+ 1185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:12.733318 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:12.734028 (+ 710us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:12.734029 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:12.734095 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:12.734098 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 10:17:12.734144 (+ 46us) client_negotiation.cc:240] Negotiation successful
0619 10:17:12.734173 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":20,"mutex_wait_us":1}
I20260619 10:17:12.734270 3048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:12.731029 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.65:38751 (local address 127.0.206.67:35437)
0619 10:17:12.731067 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0619 10:17:12.731069 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 10:17:12.731076 (+ 7us) server_negotiation.cc:407] Connection header received
0619 10:17:12.731095 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:17:12.731096 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:17:12.731105 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:17:12.731146 (+ 41us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 10:17:12.731315 (+ 169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:12.732059 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:12.733411 (+ 1352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:12.733949 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:12.734022 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:12.734029 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0619 10:17:12.734166 (+ 137us) server_negotiation.cc:299] Negotiation successful
0619 10:17:12.734209 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260619 10:17:12.734351 3050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:12.731116 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.65:39731 (local address 127.0.206.66:41427)
0619 10:17:12.731179 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0619 10:17:12.731181 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 10:17:12.731298 (+ 117us) server_negotiation.cc:407] Connection header received
0619 10:17:12.731351 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:17:12.731352 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:17:12.731360 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:17:12.731429 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 10:17:12.731509 (+ 80us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:12.732156 (+ 647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:12.733418 (+ 1262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:12.733949 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:12.733995 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:12.734004 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0619 10:17:12.734244 (+ 240us) server_negotiation.cc:299] Negotiation successful
0619 10:17:12.734284 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260619 10:17:12.734493 3060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:12.730950 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.66:41427 (local address 127.0.206.65:39731)
0619 10:17:12.731224 (+ 274us) negotiation.cc:107] Waiting for socket to connect
0619 10:17:12.731233 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0619 10:17:12.731289 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 10:17:12.731429 (+ 140us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 10:17:12.731430 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 10:17:12.731431 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 10:17:12.731473 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:12.731474 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:12.732220 (+ 746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:12.732221 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:12.733318 (+ 1097us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:12.733319 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:12.734062 (+ 743us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:12.734062 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:12.734164 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:12.734168 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 10:17:12.734220 (+ 52us) client_negotiation.cc:240] Negotiation successful
0619 10:17:12.734256 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":256,"mutex_wait_us":33,"thread_start_us":97,"threads_started":1}
I20260619 10:17:12.734517 2985 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7535de8b335d4947bcc11e3c6aa0d418" candidate_uuid: "b3a2757b35924642b12335f33832df9d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2b2d8effb39f4473b8bb107821ef1c81" is_pre_election: true
I20260619 10:17:12.734655 2985 raft_consensus.cc:2454] T 7535de8b335d4947bcc11e3c6aa0d418 P 2b2d8effb39f4473b8bb107821ef1c81 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b3a2757b35924642b12335f33832df9d in term 0.
I20260619 10:17:12.734781 2845 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7535de8b335d4947bcc11e3c6aa0d418" candidate_uuid: "b3a2757b35924642b12335f33832df9d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d869af37b51740718e3ea96f0972bc81" is_pre_election: true
I20260619 10:17:12.734892 2845 raft_consensus.cc:2454] T 7535de8b335d4947bcc11e3c6aa0d418 P d869af37b51740718e3ea96f0972bc81 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b3a2757b35924642b12335f33832df9d in term 0.
I20260619 10:17:12.734877 2638 leader_election.cc:302] T 7535de8b335d4947bcc11e3c6aa0d418 P b3a2757b35924642b12335f33832df9d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2b2d8effb39f4473b8bb107821ef1c81, b3a2757b35924642b12335f33832df9d; no voters:
I20260619 10:17:12.734984 3058 raft_consensus.cc:2781] T 7535de8b335d4947bcc11e3c6aa0d418 P b3a2757b35924642b12335f33832df9d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 10:17:12.735037 3058 raft_consensus.cc:482] T 7535de8b335d4947bcc11e3c6aa0d418 P b3a2757b35924642b12335f33832df9d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 10:17:12.735060 3058 raft_consensus.cc:3037] T 7535de8b335d4947bcc11e3c6aa0d418 P b3a2757b35924642b12335f33832df9d [term 0 FOLLOWER]: Advancing to term 1
I20260619 10:17:12.735649 3058 raft_consensus.cc:504] T 7535de8b335d4947bcc11e3c6aa0d418 P b3a2757b35924642b12335f33832df9d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3a2757b35924642b12335f33832df9d" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 35095 } } peers { permanent_uuid: "2b2d8effb39f4473b8bb107821ef1c81" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 35437 } } peers { permanent_uuid: "d869af37b51740718e3ea96f0972bc81" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 41427 } }
I20260619 10:17:12.735774 3058 leader_election.cc:288] T 7535de8b335d4947bcc11e3c6aa0d418 P b3a2757b35924642b12335f33832df9d [CANDIDATE]: Term 1 election: Requested vote from peers 2b2d8effb39f4473b8bb107821ef1c81 (127.0.206.67:35437), d869af37b51740718e3ea96f0972bc81 (127.0.206.66:41427)
I20260619 10:17:12.735888 2985 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7535de8b335d4947bcc11e3c6aa0d418" candidate_uuid: "b3a2757b35924642b12335f33832df9d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2b2d8effb39f4473b8bb107821ef1c81"
I20260619 10:17:12.735953 2985 raft_consensus.cc:3037] T 7535de8b335d4947bcc11e3c6aa0d418 P 2b2d8effb39f4473b8bb107821ef1c81 [term 0 FOLLOWER]: Advancing to term 1
I20260619 10:17:12.736553 2985 raft_consensus.cc:2454] T 7535de8b335d4947bcc11e3c6aa0d418 P 2b2d8effb39f4473b8bb107821ef1c81 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b3a2757b35924642b12335f33832df9d in term 1.
I20260619 10:17:12.736694 2638 leader_election.cc:302] T 7535de8b335d4947bcc11e3c6aa0d418 P b3a2757b35924642b12335f33832df9d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2b2d8effb39f4473b8bb107821ef1c81, b3a2757b35924642b12335f33832df9d; no voters:
I20260619 10:17:12.736703 2845 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7535de8b335d4947bcc11e3c6aa0d418" candidate_uuid: "b3a2757b35924642b12335f33832df9d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d869af37b51740718e3ea96f0972bc81"
I20260619 10:17:12.736779 2845 raft_consensus.cc:3037] T 7535de8b335d4947bcc11e3c6aa0d418 P d869af37b51740718e3ea96f0972bc81 [term 0 FOLLOWER]: Advancing to term 1
I20260619 10:17:12.736826 3058 raft_consensus.cc:2781] T 7535de8b335d4947bcc11e3c6aa0d418 P b3a2757b35924642b12335f33832df9d [term 1 FOLLOWER]: Leader election won for term 1
I20260619 10:17:12.736989 3058 raft_consensus.cc:686] T 7535de8b335d4947bcc11e3c6aa0d418 P b3a2757b35924642b12335f33832df9d [term 1 LEADER]: Becoming Leader. State: Replica: b3a2757b35924642b12335f33832df9d, State: Running, Role: LEADER
I20260619 10:17:12.737071 3058 consensus_queue.cc:237] T 7535de8b335d4947bcc11e3c6aa0d418 P b3a2757b35924642b12335f33832df9d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3a2757b35924642b12335f33832df9d" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 35095 } } peers { permanent_uuid: "2b2d8effb39f4473b8bb107821ef1c81" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 35437 } } peers { permanent_uuid: "d869af37b51740718e3ea96f0972bc81" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 41427 } }
I20260619 10:17:12.737573 2845 raft_consensus.cc:2454] T 7535de8b335d4947bcc11e3c6aa0d418 P d869af37b51740718e3ea96f0972bc81 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b3a2757b35924642b12335f33832df9d in term 1.
I20260619 10:17:12.737716 2547 catalog_manager.cc:5495] T 7535de8b335d4947bcc11e3c6aa0d418 P b3a2757b35924642b12335f33832df9d reported cstate change: term changed from 0 to 1, leader changed from <none> to b3a2757b35924642b12335f33832df9d (127.0.206.65). New cstate: current_term: 1 leader_uuid: "b3a2757b35924642b12335f33832df9d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3a2757b35924642b12335f33832df9d" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 35095 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2b2d8effb39f4473b8bb107821ef1c81" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 35437 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d869af37b51740718e3ea96f0972bc81" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 41427 } health_report { overall_health: UNKNOWN } } }
Jun 19 10:17:12 dist-test-slave-6pt2 krb5kdc[2503](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864232, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.126@KRBTEST.COM for kudu/127.0.206.126@KRBTEST.COM
W20260619 10:17:12.773499 2892 tablet.cc:2369] T 7535de8b335d4947bcc11e3c6aa0d418 P d869af37b51740718e3ea96f0972bc81: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 10:17:12.774060 3068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:12.767376 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:36165 (local address 127.0.0.1:36492)
0619 10:17:12.767584 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0619 10:17:12.767597 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 10:17:12.767687 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 10:17:12.767841 (+ 154us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 10:17:12.767842 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 10:17:12.768145 (+ 303us) client_negotiation.cc:189] Negotiated authn=SASL
0619 10:17:12.768187 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:12.768189 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:12.768837 (+ 648us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:12.768838 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:12.769348 (+ 510us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:12.769349 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:12.769716 (+ 367us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:12.769716 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:12.769774 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:12.770237 (+ 463us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 10:17:12.770242 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 10:17:12.771563 (+ 1321us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 10:17:12.773098 (+ 1535us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 10:17:12.773103 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 10:17:12.773105 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 10:17:12.773273 (+ 168us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 10:17:12.773449 (+ 176us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 10:17:12.773450 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 10:17:12.773450 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 10:17:12.773514 (+ 64us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 10:17:12.773782 (+ 268us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 10:17:12.773788 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 10:17:12.773839 (+ 51us) client_negotiation.cc:769] Sending connection context
0619 10:17:12.773896 (+ 57us) client_negotiation.cc:240] Negotiation successful
0619 10:17:12.773993 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":184,"thread_start_us":123,"threads_started":1}
I20260619 10:17:12.774173 2598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:12.767475 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36492 (local address 127.0.206.126:36165)
0619 10:17:12.767550 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0619 10:17:12.767553 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 10:17:12.767667 (+ 114us) server_negotiation.cc:407] Connection header received
0619 10:17:12.767746 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:17:12.767747 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:17:12.767756 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:17:12.767828 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:17:12.768271 (+ 443us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:12.768755 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:12.769440 (+ 685us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:12.769617 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:12.769702 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:12.771654 (+ 1952us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 10:17:12.771656 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 10:17:12.771656 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 10:17:12.771662 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 10:17:12.773028 (+ 1366us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:17:12.773344 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:17:12.773344 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:17:12.773345 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:17:12.773392 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:17:12.773602 (+ 210us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:17:12.773602 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:17:12.773603 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:17:12.773721 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 10:17:12.773771 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0619 10:17:12.773944 (+ 173us) server_negotiation.cc:299] Negotiation successful
0619 10:17:12.774049 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":16,"server-negotiator.queue_time_us":50}
I20260619 10:17:12.776060 2547 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.0.206.126@KRBTEST.COM'} at 127.0.0.1:36492:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260619 10:17:12.776288 2547 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 10:17:12.777731 2825 tablet_service.cc:1467] Processing CreateTablet for tablet 6901003618e84e9a91f53f0e5136e6fb (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=d7f1978ef52d4395b5f6325187543fe7]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260619 10:17:12.777777 2965 tablet_service.cc:1467] Processing CreateTablet for tablet 6901003618e84e9a91f53f0e5136e6fb (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=d7f1978ef52d4395b5f6325187543fe7]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260619 10:17:12.777921 2825 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6901003618e84e9a91f53f0e5136e6fb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 10:17:12.777877 2685 tablet_service.cc:1467] Processing CreateTablet for tablet 6901003618e84e9a91f53f0e5136e6fb (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=d7f1978ef52d4395b5f6325187543fe7]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260619 10:17:12.777937 2965 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6901003618e84e9a91f53f0e5136e6fb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 10:17:12.778007 2685 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6901003618e84e9a91f53f0e5136e6fb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 10:17:12.779352 3051 tablet_bootstrap.cc:492] T 6901003618e84e9a91f53f0e5136e6fb P 2b2d8effb39f4473b8bb107821ef1c81: Bootstrap starting.
I20260619 10:17:12.779461 3055 tablet_bootstrap.cc:492] T 6901003618e84e9a91f53f0e5136e6fb P d869af37b51740718e3ea96f0972bc81: Bootstrap starting.
I20260619 10:17:12.779462 3052 tablet_bootstrap.cc:492] T 6901003618e84e9a91f53f0e5136e6fb P b3a2757b35924642b12335f33832df9d: Bootstrap starting.
I20260619 10:17:12.779865 3051 tablet_bootstrap.cc:654] T 6901003618e84e9a91f53f0e5136e6fb P 2b2d8effb39f4473b8bb107821ef1c81: Neither blocks nor log segments found. Creating new log.
I20260619 10:17:12.779923 3052 tablet_bootstrap.cc:654] T 6901003618e84e9a91f53f0e5136e6fb P b3a2757b35924642b12335f33832df9d: Neither blocks nor log segments found. Creating new log.
I20260619 10:17:12.780157 3055 tablet_bootstrap.cc:654] T 6901003618e84e9a91f53f0e5136e6fb P d869af37b51740718e3ea96f0972bc81: Neither blocks nor log segments found. Creating new log.
I20260619 10:17:12.780632 3052 tablet_bootstrap.cc:492] T 6901003618e84e9a91f53f0e5136e6fb P b3a2757b35924642b12335f33832df9d: No bootstrap required, opened a new log
I20260619 10:17:12.780683 3052 ts_tablet_manager.cc:1397] T 6901003618e84e9a91f53f0e5136e6fb P b3a2757b35924642b12335f33832df9d: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260619 10:17:12.780833 3052 raft_consensus.cc:348] T 6901003618e84e9a91f53f0e5136e6fb P b3a2757b35924642b12335f33832df9d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3a2757b35924642b12335f33832df9d" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 35095 } } peers { permanent_uuid: "d869af37b51740718e3ea96f0972bc81" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 41427 } } peers { permanent_uuid: "2b2d8effb39f4473b8bb107821ef1c81" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 35437 } }
I20260619 10:17:12.780848 3055 tablet_bootstrap.cc:492] T 6901003618e84e9a91f53f0e5136e6fb P d869af37b51740718e3ea96f0972bc81: No bootstrap required, opened a new log
I20260619 10:17:12.780903 3052 raft_consensus.cc:374] T 6901003618e84e9a91f53f0e5136e6fb P b3a2757b35924642b12335f33832df9d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 10:17:12.780936 3052 raft_consensus.cc:729] T 6901003618e84e9a91f53f0e5136e6fb P b3a2757b35924642b12335f33832df9d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b3a2757b35924642b12335f33832df9d, State: Initialized, Role: FOLLOWER
I20260619 10:17:12.780927 3055 ts_tablet_manager.cc:1397] T 6901003618e84e9a91f53f0e5136e6fb P d869af37b51740718e3ea96f0972bc81: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260619 10:17:12.780978 3052 consensus_queue.cc:260] T 6901003618e84e9a91f53f0e5136e6fb P b3a2757b35924642b12335f33832df9d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3a2757b35924642b12335f33832df9d" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 35095 } } peers { permanent_uuid: "d869af37b51740718e3ea96f0972bc81" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 41427 } } peers { permanent_uuid: "2b2d8effb39f4473b8bb107821ef1c81" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 35437 } }
I20260619 10:17:12.780990 3051 tablet_bootstrap.cc:492] T 6901003618e84e9a91f53f0e5136e6fb P 2b2d8effb39f4473b8bb107821ef1c81: No bootstrap required, opened a new log
I20260619 10:17:12.781029 3051 ts_tablet_manager.cc:1397] T 6901003618e84e9a91f53f0e5136e6fb P 2b2d8effb39f4473b8bb107821ef1c81: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260619 10:17:12.781056 3052 ts_tablet_manager.cc:1428] T 6901003618e84e9a91f53f0e5136e6fb P b3a2757b35924642b12335f33832df9d: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260619 10:17:12.781075 3055 raft_consensus.cc:348] T 6901003618e84e9a91f53f0e5136e6fb P d869af37b51740718e3ea96f0972bc81 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3a2757b35924642b12335f33832df9d" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 35095 } } peers { permanent_uuid: "d869af37b51740718e3ea96f0972bc81" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 41427 } } peers { permanent_uuid: "2b2d8effb39f4473b8bb107821ef1c81" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 35437 } }
I20260619 10:17:12.781140 3055 raft_consensus.cc:374] T 6901003618e84e9a91f53f0e5136e6fb P d869af37b51740718e3ea96f0972bc81 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 10:17:12.781086 3058 tablet_replica.cc:439] T 6901003618e84e9a91f53f0e5136e6fb P b3a2757b35924642b12335f33832df9d: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3a2757b35924642b12335f33832df9d" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 35095 } } peers { permanent_uuid: "d869af37b51740718e3ea96f0972bc81" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 41427 } } peers { permanent_uuid: "2b2d8effb39f4473b8bb107821ef1c81" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 35437 } } }
I20260619 10:17:12.781167 3058 tablet_replica.cc:442] T 6901003618e84e9a91f53f0e5136e6fb P b3a2757b35924642b12335f33832df9d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 10:17:12.781159 3055 raft_consensus.cc:729] T 6901003618e84e9a91f53f0e5136e6fb P d869af37b51740718e3ea96f0972bc81 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d869af37b51740718e3ea96f0972bc81, State: Initialized, Role: FOLLOWER
I20260619 10:17:12.781172 3051 raft_consensus.cc:348] T 6901003618e84e9a91f53f0e5136e6fb P 2b2d8effb39f4473b8bb107821ef1c81 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3a2757b35924642b12335f33832df9d" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 35095 } } peers { permanent_uuid: "d869af37b51740718e3ea96f0972bc81" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 41427 } } peers { permanent_uuid: "2b2d8effb39f4473b8bb107821ef1c81" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 35437 } }
I20260619 10:17:12.781236 3051 raft_consensus.cc:374] T 6901003618e84e9a91f53f0e5136e6fb P 2b2d8effb39f4473b8bb107821ef1c81 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 10:17:12.781217 3055 consensus_queue.cc:260] T 6901003618e84e9a91f53f0e5136e6fb P d869af37b51740718e3ea96f0972bc81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3a2757b35924642b12335f33832df9d" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 35095 } } peers { permanent_uuid: "d869af37b51740718e3ea96f0972bc81" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 41427 } } peers { permanent_uuid: "2b2d8effb39f4473b8bb107821ef1c81" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 35437 } }
I20260619 10:17:12.781256 3051 raft_consensus.cc:729] T 6901003618e84e9a91f53f0e5136e6fb P 2b2d8effb39f4473b8bb107821ef1c81 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2b2d8effb39f4473b8bb107821ef1c81, State: Initialized, Role: FOLLOWER
I20260619 10:17:12.781316 3055 ts_tablet_manager.cc:1428] T 6901003618e84e9a91f53f0e5136e6fb P d869af37b51740718e3ea96f0972bc81: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260619 10:17:12.781301 3051 consensus_queue.cc:260] T 6901003618e84e9a91f53f0e5136e6fb P 2b2d8effb39f4473b8bb107821ef1c81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3a2757b35924642b12335f33832df9d" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 35095 } } peers { permanent_uuid: "d869af37b51740718e3ea96f0972bc81" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 41427 } } peers { permanent_uuid: "2b2d8effb39f4473b8bb107821ef1c81" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 35437 } }
I20260619 10:17:12.781383 3051 ts_tablet_manager.cc:1428] T 6901003618e84e9a91f53f0e5136e6fb P 2b2d8effb39f4473b8bb107821ef1c81: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260619 10:17:12.781441 3059 tablet_replica.cc:439] T 6901003618e84e9a91f53f0e5136e6fb P d869af37b51740718e3ea96f0972bc81: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3a2757b35924642b12335f33832df9d" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 35095 } } peers { permanent_uuid: "d869af37b51740718e3ea96f0972bc81" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 41427 } } peers { permanent_uuid: "2b2d8effb39f4473b8bb107821ef1c81" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 35437 } } }
I20260619 10:17:12.781443 3056 tablet_replica.cc:439] T 6901003618e84e9a91f53f0e5136e6fb P 2b2d8effb39f4473b8bb107821ef1c81: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3a2757b35924642b12335f33832df9d" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 35095 } } peers { permanent_uuid: "d869af37b51740718e3ea96f0972bc81" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 41427 } } peers { permanent_uuid: "2b2d8effb39f4473b8bb107821ef1c81" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 35437 } } }
I20260619 10:17:12.781526 3059 tablet_replica.cc:442] T 6901003618e84e9a91f53f0e5136e6fb P d869af37b51740718e3ea96f0972bc81: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 10:17:12.781531 3056 tablet_replica.cc:442] T 6901003618e84e9a91f53f0e5136e6fb P 2b2d8effb39f4473b8bb107821ef1c81: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 10:17:12.818714 3056 raft_consensus.cc:482] T 6901003618e84e9a91f53f0e5136e6fb P 2b2d8effb39f4473b8bb107821ef1c81 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 10:17:12.818831 3056 raft_consensus.cc:504] T 6901003618e84e9a91f53f0e5136e6fb P 2b2d8effb39f4473b8bb107821ef1c81 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3a2757b35924642b12335f33832df9d" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 35095 } } peers { permanent_uuid: "d869af37b51740718e3ea96f0972bc81" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 41427 } } peers { permanent_uuid: "2b2d8effb39f4473b8bb107821ef1c81" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 35437 } }
I20260619 10:17:12.819175 3056 leader_election.cc:288] T 6901003618e84e9a91f53f0e5136e6fb P 2b2d8effb39f4473b8bb107821ef1c81 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b3a2757b35924642b12335f33832df9d (127.0.206.65:35095), d869af37b51740718e3ea96f0972bc81 (127.0.206.66:41427)
I20260619 10:17:12.822105 3050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:12.819544 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.67:51439 (local address 127.0.206.66:41427)
0619 10:17:12.819583 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0619 10:17:12.819585 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 10:17:12.819720 (+ 135us) server_negotiation.cc:407] Connection header received
0619 10:17:12.819755 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:17:12.819756 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:17:12.819766 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:17:12.819820 (+ 54us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 10:17:12.819963 (+ 143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:12.820471 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:12.821395 (+ 924us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:12.821730 (+ 335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:12.821817 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:12.821826 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0619 10:17:12.821980 (+ 154us) server_negotiation.cc:299] Negotiation successful
0619 10:17:12.822022 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260619 10:17:12.822129 2924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:12.819352 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.65:35095 (local address 127.0.206.67:36565)
0619 10:17:12.819410 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0619 10:17:12.819429 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0619 10:17:12.819482 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 10:17:12.819650 (+ 168us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 10:17:12.819651 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 10:17:12.819657 (+ 6us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 10:17:12.819749 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:12.819751 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:12.820461 (+ 710us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:12.820462 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:12.821263 (+ 801us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:12.821265 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:12.821827 (+ 562us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:12.821828 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:12.821925 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:12.821928 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 10:17:12.822001 (+ 73us) client_negotiation.cc:240] Negotiation successful
0619 10:17:12.822033 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36,"mutex_wait_us":22}
I20260619 10:17:12.822139 3047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:12.819462 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.67:36565 (local address 127.0.206.65:35095)
0619 10:17:12.819515 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0619 10:17:12.819518 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 10:17:12.819528 (+ 10us) server_negotiation.cc:407] Connection header received
0619 10:17:12.819557 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:17:12.819557 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:17:12.819566 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:17:12.819641 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 10:17:12.819847 (+ 206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:12.820361 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:12.821351 (+ 990us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:12.821730 (+ 379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:12.821836 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:12.821844 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0619 10:17:12.822015 (+ 171us) server_negotiation.cc:299] Negotiation successful
0619 10:17:12.822066 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260619 10:17:12.822371 3072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:12.819462 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.66:41427 (local address 127.0.206.67:51439)
0619 10:17:12.819642 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0619 10:17:12.819653 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 10:17:12.819710 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 10:17:12.819834 (+ 124us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 10:17:12.819835 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 10:17:12.819837 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 10:17:12.819904 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:12.819905 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:12.820546 (+ 641us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:12.820547 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:12.821273 (+ 726us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:12.821275 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:12.821832 (+ 557us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:12.821833 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:12.821911 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:12.821914 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 10:17:12.821956 (+ 42us) client_negotiation.cc:240] Negotiation successful
0619 10:17:12.822012 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":162,"thread_start_us":110,"threads_started":1}
I20260619 10:17:12.822474 2705 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6901003618e84e9a91f53f0e5136e6fb" candidate_uuid: "2b2d8effb39f4473b8bb107821ef1c81" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b3a2757b35924642b12335f33832df9d" is_pre_election: true
I20260619 10:17:12.822625 2705 raft_consensus.cc:2454] T 6901003618e84e9a91f53f0e5136e6fb P b3a2757b35924642b12335f33832df9d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2b2d8effb39f4473b8bb107821ef1c81 in term 0.
I20260619 10:17:12.822639 2845 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6901003618e84e9a91f53f0e5136e6fb" candidate_uuid: "2b2d8effb39f4473b8bb107821ef1c81" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d869af37b51740718e3ea96f0972bc81" is_pre_election: true
I20260619 10:17:12.822728 2845 raft_consensus.cc:2454] T 6901003618e84e9a91f53f0e5136e6fb P d869af37b51740718e3ea96f0972bc81 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2b2d8effb39f4473b8bb107821ef1c81 in term 0.
I20260619 10:17:12.822818 2916 leader_election.cc:302] T 6901003618e84e9a91f53f0e5136e6fb P 2b2d8effb39f4473b8bb107821ef1c81 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2b2d8effb39f4473b8bb107821ef1c81, b3a2757b35924642b12335f33832df9d; no voters:
I20260619 10:17:12.822938 3056 raft_consensus.cc:2781] T 6901003618e84e9a91f53f0e5136e6fb P 2b2d8effb39f4473b8bb107821ef1c81 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 10:17:12.822974 3056 raft_consensus.cc:482] T 6901003618e84e9a91f53f0e5136e6fb P 2b2d8effb39f4473b8bb107821ef1c81 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 10:17:12.822988 3056 raft_consensus.cc:3037] T 6901003618e84e9a91f53f0e5136e6fb P 2b2d8effb39f4473b8bb107821ef1c81 [term 0 FOLLOWER]: Advancing to term 1
I20260619 10:17:12.823619 3056 raft_consensus.cc:504] T 6901003618e84e9a91f53f0e5136e6fb P 2b2d8effb39f4473b8bb107821ef1c81 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3a2757b35924642b12335f33832df9d" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 35095 } } peers { permanent_uuid: "d869af37b51740718e3ea96f0972bc81" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 41427 } } peers { permanent_uuid: "2b2d8effb39f4473b8bb107821ef1c81" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 35437 } }
I20260619 10:17:12.823742 3056 leader_election.cc:288] T 6901003618e84e9a91f53f0e5136e6fb P 2b2d8effb39f4473b8bb107821ef1c81 [CANDIDATE]: Term 1 election: Requested vote from peers b3a2757b35924642b12335f33832df9d (127.0.206.65:35095), d869af37b51740718e3ea96f0972bc81 (127.0.206.66:41427)
I20260619 10:17:12.823904 2705 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6901003618e84e9a91f53f0e5136e6fb" candidate_uuid: "2b2d8effb39f4473b8bb107821ef1c81" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b3a2757b35924642b12335f33832df9d"
I20260619 10:17:12.823900 2845 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6901003618e84e9a91f53f0e5136e6fb" candidate_uuid: "2b2d8effb39f4473b8bb107821ef1c81" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d869af37b51740718e3ea96f0972bc81"
I20260619 10:17:12.823967 2705 raft_consensus.cc:3037] T 6901003618e84e9a91f53f0e5136e6fb P b3a2757b35924642b12335f33832df9d [term 0 FOLLOWER]: Advancing to term 1
I20260619 10:17:12.823968 2845 raft_consensus.cc:3037] T 6901003618e84e9a91f53f0e5136e6fb P d869af37b51740718e3ea96f0972bc81 [term 0 FOLLOWER]: Advancing to term 1
I20260619 10:17:12.824683 2845 raft_consensus.cc:2454] T 6901003618e84e9a91f53f0e5136e6fb P d869af37b51740718e3ea96f0972bc81 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2b2d8effb39f4473b8bb107821ef1c81 in term 1.
I20260619 10:17:12.824702 2705 raft_consensus.cc:2454] T 6901003618e84e9a91f53f0e5136e6fb P b3a2757b35924642b12335f33832df9d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2b2d8effb39f4473b8bb107821ef1c81 in term 1.
I20260619 10:17:12.824832 2916 leader_election.cc:302] T 6901003618e84e9a91f53f0e5136e6fb P 2b2d8effb39f4473b8bb107821ef1c81 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2b2d8effb39f4473b8bb107821ef1c81, d869af37b51740718e3ea96f0972bc81; no voters:
I20260619 10:17:12.824935 3056 raft_consensus.cc:2781] T 6901003618e84e9a91f53f0e5136e6fb P 2b2d8effb39f4473b8bb107821ef1c81 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 10:17:12.825120 3056 raft_consensus.cc:686] T 6901003618e84e9a91f53f0e5136e6fb P 2b2d8effb39f4473b8bb107821ef1c81 [term 1 LEADER]: Becoming Leader. State: Replica: 2b2d8effb39f4473b8bb107821ef1c81, State: Running, Role: LEADER
I20260619 10:17:12.825234 3056 consensus_queue.cc:237] T 6901003618e84e9a91f53f0e5136e6fb P 2b2d8effb39f4473b8bb107821ef1c81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3a2757b35924642b12335f33832df9d" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 35095 } } peers { permanent_uuid: "d869af37b51740718e3ea96f0972bc81" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 41427 } } peers { permanent_uuid: "2b2d8effb39f4473b8bb107821ef1c81" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 35437 } }
I20260619 10:17:12.825697 3073 tablet_replica.cc:439] T 6901003618e84e9a91f53f0e5136e6fb P 2b2d8effb39f4473b8bb107821ef1c81: TxnStatusTablet state changed. Reason: New leader 2b2d8effb39f4473b8bb107821ef1c81. Latest consensus state: current_term: 1 leader_uuid: "2b2d8effb39f4473b8bb107821ef1c81" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3a2757b35924642b12335f33832df9d" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 35095 } } peers { permanent_uuid: "d869af37b51740718e3ea96f0972bc81" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 41427 } } peers { permanent_uuid: "2b2d8effb39f4473b8bb107821ef1c81" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 35437 } } }
I20260619 10:17:12.825784 3073 tablet_replica.cc:442] T 6901003618e84e9a91f53f0e5136e6fb P 2b2d8effb39f4473b8bb107821ef1c81: This TxnStatusTablet replica's current role is: LEADER
I20260619 10:17:12.825963 3075 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260619 10:17:12.826073 3075 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260619 10:17:12.826015 2548 catalog_manager.cc:5495] T 6901003618e84e9a91f53f0e5136e6fb P 2b2d8effb39f4473b8bb107821ef1c81 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2b2d8effb39f4473b8bb107821ef1c81 (127.0.206.67). New cstate: current_term: 1 leader_uuid: "2b2d8effb39f4473b8bb107821ef1c81" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3a2757b35924642b12335f33832df9d" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 35095 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d869af37b51740718e3ea96f0972bc81" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 41427 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2b2d8effb39f4473b8bb107821ef1c81" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 35437 } health_report { overall_health: HEALTHY } } }
I20260619 10:17:12.840890 3048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:12.838495 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48630 (local address 127.0.206.67:35437)
0619 10:17:12.838537 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0619 10:17:12.838539 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 10:17:12.838562 (+ 23us) server_negotiation.cc:407] Connection header received
0619 10:17:12.838676 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:17:12.838677 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:17:12.838693 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:17:12.838771 (+ 78us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 10:17:12.838894 (+ 123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:12.839341 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:12.840080 (+ 739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:12.840292 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:12.840366 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:12.840569 (+ 203us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 10:17:12.840665 (+ 96us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 10:17:12.840750 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0619 10:17:12.840791 (+ 41us) server_negotiation.cc:299] Negotiation successful
0619 10:17:12.840809 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260619 10:17:12.840915 3068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:12.838439 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.67:35437 (local address 127.0.0.1:48630)
0619 10:17:12.838515 (+ 76us) negotiation.cc:107] Waiting for socket to connect
0619 10:17:12.838526 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 10:17:12.838582 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 10:17:12.838778 (+ 196us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 10:17:12.838779 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 10:17:12.838780 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0619 10:17:12.838832 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:12.838834 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:12.839422 (+ 588us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:12.839423 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:12.839989 (+ 566us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:12.839990 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:12.840367 (+ 377us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:12.840368 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:12.840436 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:12.840457 (+ 21us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0619 10:17:12.840729 (+ 272us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0619 10:17:12.840738 (+ 9us) client_negotiation.cc:769] Sending connection context
0619 10:17:12.840786 (+ 48us) client_negotiation.cc:240] Negotiation successful
0619 10:17:12.840837 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":45}
I20260619 10:17:12.842870 2845 raft_consensus.cc:1261] T 6901003618e84e9a91f53f0e5136e6fb P d869af37b51740718e3ea96f0972bc81 [term 1 FOLLOWER]: Refusing update from remote peer 2b2d8effb39f4473b8bb107821ef1c81: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 10:17:12.842875 2705 raft_consensus.cc:1261] T 6901003618e84e9a91f53f0e5136e6fb P b3a2757b35924642b12335f33832df9d [term 1 FOLLOWER]: Refusing update from remote peer 2b2d8effb39f4473b8bb107821ef1c81: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 10:17:12.843142 3056 consensus_queue.cc:1035] T 6901003618e84e9a91f53f0e5136e6fb P 2b2d8effb39f4473b8bb107821ef1c81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b3a2757b35924642b12335f33832df9d" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 35095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 10:17:12.843298 3056 consensus_queue.cc:1035] T 6901003618e84e9a91f53f0e5136e6fb P 2b2d8effb39f4473b8bb107821ef1c81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d869af37b51740718e3ea96f0972bc81" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 41427 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 10:17:12.844390 3058 tablet_replica.cc:439] T 6901003618e84e9a91f53f0e5136e6fb P b3a2757b35924642b12335f33832df9d: TxnStatusTablet state changed. Reason: New leader 2b2d8effb39f4473b8bb107821ef1c81. Latest consensus state: current_term: 1 leader_uuid: "2b2d8effb39f4473b8bb107821ef1c81" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3a2757b35924642b12335f33832df9d" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 35095 } } peers { permanent_uuid: "d869af37b51740718e3ea96f0972bc81" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 41427 } } peers { permanent_uuid: "2b2d8effb39f4473b8bb107821ef1c81" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 35437 } } }
I20260619 10:17:12.844487 3058 tablet_replica.cc:442] T 6901003618e84e9a91f53f0e5136e6fb P b3a2757b35924642b12335f33832df9d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 10:17:12.844523 3059 tablet_replica.cc:439] T 6901003618e84e9a91f53f0e5136e6fb P d869af37b51740718e3ea96f0972bc81: TxnStatusTablet state changed. Reason: New leader 2b2d8effb39f4473b8bb107821ef1c81. Latest consensus state: current_term: 1 leader_uuid: "2b2d8effb39f4473b8bb107821ef1c81" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3a2757b35924642b12335f33832df9d" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 35095 } } peers { permanent_uuid: "d869af37b51740718e3ea96f0972bc81" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 41427 } } peers { permanent_uuid: "2b2d8effb39f4473b8bb107821ef1c81" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 35437 } } }
I20260619 10:17:12.844604 3059 tablet_replica.cc:442] T 6901003618e84e9a91f53f0e5136e6fb P d869af37b51740718e3ea96f0972bc81: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 10:17:12.845371 3056 tablet_replica.cc:439] T 6901003618e84e9a91f53f0e5136e6fb P 2b2d8effb39f4473b8bb107821ef1c81: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "2b2d8effb39f4473b8bb107821ef1c81" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3a2757b35924642b12335f33832df9d" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 35095 } } peers { permanent_uuid: "d869af37b51740718e3ea96f0972bc81" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 41427 } } peers { permanent_uuid: "2b2d8effb39f4473b8bb107821ef1c81" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 35437 } } }
I20260619 10:17:12.845448 3056 tablet_replica.cc:442] T 6901003618e84e9a91f53f0e5136e6fb P 2b2d8effb39f4473b8bb107821ef1c81: This TxnStatusTablet replica's current role is: LEADER
I20260619 10:17:12.845788 3056 tablet_replica.cc:439] T 6901003618e84e9a91f53f0e5136e6fb P 2b2d8effb39f4473b8bb107821ef1c81: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "2b2d8effb39f4473b8bb107821ef1c81" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3a2757b35924642b12335f33832df9d" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 35095 } } peers { permanent_uuid: "d869af37b51740718e3ea96f0972bc81" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 41427 } } peers { permanent_uuid: "2b2d8effb39f4473b8bb107821ef1c81" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 35437 } } }
I20260619 10:17:12.845863 3056 tablet_replica.cc:442] T 6901003618e84e9a91f53f0e5136e6fb P 2b2d8effb39f4473b8bb107821ef1c81: This TxnStatusTablet replica's current role is: LEADER
I20260619 10:17:12.845947 3058 tablet_replica.cc:439] T 6901003618e84e9a91f53f0e5136e6fb P b3a2757b35924642b12335f33832df9d: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "2b2d8effb39f4473b8bb107821ef1c81" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3a2757b35924642b12335f33832df9d" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 35095 } } peers { permanent_uuid: "d869af37b51740718e3ea96f0972bc81" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 41427 } } peers { permanent_uuid: "2b2d8effb39f4473b8bb107821ef1c81" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 35437 } } }
I20260619 10:17:12.846015 3058 tablet_replica.cc:442] T 6901003618e84e9a91f53f0e5136e6fb P b3a2757b35924642b12335f33832df9d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 10:17:12.846155 3079 mvcc.cc:205] Tried to move back new op lower bound from 7298515897721843712 to 7298515897652719616. Current Snapshot: MvccSnapshot[applied={T|T < 7298515897721843712}]
I20260619 10:17:12.846413 3059 tablet_replica.cc:439] T 6901003618e84e9a91f53f0e5136e6fb P d869af37b51740718e3ea96f0972bc81: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "2b2d8effb39f4473b8bb107821ef1c81" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3a2757b35924642b12335f33832df9d" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 35095 } } peers { permanent_uuid: "d869af37b51740718e3ea96f0972bc81" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 41427 } } peers { permanent_uuid: "2b2d8effb39f4473b8bb107821ef1c81" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 35437 } } }
I20260619 10:17:12.846498 3059 tablet_replica.cc:442] T 6901003618e84e9a91f53f0e5136e6fb P d869af37b51740718e3ea96f0972bc81: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 10:17:12.846828 3080 mvcc.cc:205] Tried to move back new op lower bound from 7298515897721843712 to 7298515897652719616. Current Snapshot: MvccSnapshot[applied={T|T < 7298515897721843712}]
I20260619 10:17:12.850664 3047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:12.848057 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57330 (local address 127.0.206.65:35095)
0619 10:17:12.848089 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0619 10:17:12.848091 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 10:17:12.848099 (+ 8us) server_negotiation.cc:407] Connection header received
0619 10:17:12.848162 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:17:12.848162 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:17:12.848174 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:17:12.848228 (+ 54us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 10:17:12.848332 (+ 104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:12.848773 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:12.849911 (+ 1138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:12.850090 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:12.850141 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:12.850304 (+ 163us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 10:17:12.850384 (+ 80us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 10:17:12.850512 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0619 10:17:12.850529 (+ 17us) server_negotiation.cc:299] Negotiation successful
0619 10:17:12.850550 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260619 10:17:12.854868 3048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:12.852768 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.65:48669 (local address 127.0.206.67:35437)
0619 10:17:12.852805 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0619 10:17:12.852807 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 10:17:12.852815 (+ 8us) server_negotiation.cc:407] Connection header received
0619 10:17:12.852878 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:17:12.852879 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:17:12.852888 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:17:12.852939 (+ 51us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 10:17:12.853034 (+ 95us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:12.853454 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:12.854158 (+ 704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:12.854345 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:12.854423 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:12.854640 (+ 217us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 10:17:12.854672 (+ 32us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 10:17:12.854739 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0619 10:17:12.854794 (+ 55us) server_negotiation.cc:299] Negotiation successful
0619 10:17:12.854810 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260619 10:17:12.854900 2757 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:12.852717 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.67:35437 (local address 127.0.206.65:48669)
0619 10:17:12.852770 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0619 10:17:12.852780 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 10:17:12.852821 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 10:17:12.852942 (+ 121us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 10:17:12.852942 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 10:17:12.852943 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0619 10:17:12.852990 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:12.852992 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:12.853533 (+ 541us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:12.853534 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:12.854066 (+ 532us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:12.854067 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:12.854425 (+ 358us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:12.854426 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:12.854490 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:12.854502 (+ 12us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0619 10:17:12.854744 (+ 242us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0619 10:17:12.854746 (+ 2us) client_negotiation.cc:769] Sending connection context
0619 10:17:12.854792 (+ 46us) client_negotiation.cc:240] Negotiation successful
0619 10:17:12.854834 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":23}
I20260619 10:17:12.856894 2845 raft_consensus.cc:1261] T 7535de8b335d4947bcc11e3c6aa0d418 P d869af37b51740718e3ea96f0972bc81 [term 1 FOLLOWER]: Refusing update from remote peer b3a2757b35924642b12335f33832df9d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 10:17:12.856992 2985 raft_consensus.cc:1261] T 7535de8b335d4947bcc11e3c6aa0d418 P 2b2d8effb39f4473b8bb107821ef1c81 [term 1 FOLLOWER]: Refusing update from remote peer b3a2757b35924642b12335f33832df9d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 10:17:12.857112 3058 consensus_queue.cc:1035] T 7535de8b335d4947bcc11e3c6aa0d418 P b3a2757b35924642b12335f33832df9d [LEADER]: Connected to new peer: Peer: permanent_uuid: "d869af37b51740718e3ea96f0972bc81" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 41427 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 10:17:12.857218 3061 consensus_queue.cc:1035] T 7535de8b335d4947bcc11e3c6aa0d418 P b3a2757b35924642b12335f33832df9d [LEADER]: Connected to new peer: Peer: permanent_uuid: "2b2d8effb39f4473b8bb107821ef1c81" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 35437 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 10:17:12.858728 3078 mvcc.cc:205] Tried to move back new op lower bound from 7298515897780101120 to 7298515897291341824. Current Snapshot: MvccSnapshot[applied={T|T < 7298515897780101120}]
I20260619 10:17:12.864692 3047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:12.862547 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.67:53673 (local address 127.0.206.65:35095)
0619 10:17:12.862585 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0619 10:17:12.862587 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 10:17:12.862613 (+ 26us) server_negotiation.cc:407] Connection header received
0619 10:17:12.862633 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:17:12.862633 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:17:12.862641 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:17:12.862685 (+ 44us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 10:17:12.862843 (+ 158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:12.863272 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:12.864028 (+ 756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:12.864219 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:12.864296 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:12.864469 (+ 173us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 10:17:12.864497 (+ 28us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 10:17:12.864561 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0619 10:17:12.864617 (+ 56us) server_negotiation.cc:299] Negotiation successful
0619 10:17:12.864633 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260619 10:17:12.864717 3037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:12.862447 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.65:35095 (local address 127.0.206.67:53673)
0619 10:17:12.862500 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0619 10:17:12.862511 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 10:17:12.862557 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 10:17:12.862721 (+ 164us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 10:17:12.862722 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 10:17:12.862723 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0619 10:17:12.862775 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:12.862776 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:12.863348 (+ 572us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:12.863349 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:12.863931 (+ 582us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:12.863933 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:12.864297 (+ 364us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:12.864298 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:12.864367 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:12.864384 (+ 17us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0619 10:17:12.864561 (+ 177us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0619 10:17:12.864563 (+ 2us) client_negotiation.cc:769] Sending connection context
0619 10:17:12.864614 (+ 51us) client_negotiation.cc:240] Negotiation successful
0619 10:17:12.864652 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":19}
I20260619 10:17:12.871824 2548 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36476:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 10:17:12.871919 2548 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36476:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 10:17:12.872653 2548 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 28ae00894a9046c291da0695c210585d: Sending DeleteTablet for 3 replicas of tablet 7535de8b335d4947bcc11e3c6aa0d418
I20260619 10:17:12.872915 2965 tablet_service.cc:1514] Processing DeleteTablet for tablet 7535de8b335d4947bcc11e3c6aa0d418 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 10:17:12 UTC) from {username='kudu', principal='kudu/127.0.206.126@KRBTEST.COM'} at 127.0.0.1:48628
I20260619 10:17:12.873211 2685 tablet_service.cc:1514] Processing DeleteTablet for tablet 7535de8b335d4947bcc11e3c6aa0d418 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 10:17:12 UTC) from {username='kudu', principal='kudu/127.0.206.126@KRBTEST.COM'} at 127.0.0.1:57326
I20260619 10:17:12.873267 2825 tablet_service.cc:1514] Processing DeleteTablet for tablet 7535de8b335d4947bcc11e3c6aa0d418 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 10:17:12 UTC) from {username='kudu', principal='kudu/127.0.206.126@KRBTEST.COM'} at 127.0.0.1:51598
I20260619 10:17:12.873468 2548 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36476:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 10:17:12.873603 2548 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 10:17:12.873855 3096 tablet_replica.cc:330] T 7535de8b335d4947bcc11e3c6aa0d418 P 2b2d8effb39f4473b8bb107821ef1c81: stopping tablet replica
I20260619 10:17:12.874686 2685 tablet_service.cc:1467] Processing CreateTablet for tablet 37abc6a50e6249c5a68fd3ddd9020f3a (DEFAULT_TABLE table=test-table [id=45aed87beac242acbf0f7f68c3703723]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 10:17:12.874704 2965 tablet_service.cc:1467] Processing CreateTablet for tablet 37abc6a50e6249c5a68fd3ddd9020f3a (DEFAULT_TABLE table=test-table [id=45aed87beac242acbf0f7f68c3703723]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 10:17:12.874823 2965 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 37abc6a50e6249c5a68fd3ddd9020f3a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 10:17:12.874851 2685 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 37abc6a50e6249c5a68fd3ddd9020f3a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 10:17:12.876024 3052 tablet_bootstrap.cc:492] T 37abc6a50e6249c5a68fd3ddd9020f3a P b3a2757b35924642b12335f33832df9d: Bootstrap starting.
I20260619 10:17:12.876132 3051 tablet_bootstrap.cc:492] T 37abc6a50e6249c5a68fd3ddd9020f3a P 2b2d8effb39f4473b8bb107821ef1c81: Bootstrap starting.
I20260619 10:17:12.876452 3052 tablet_bootstrap.cc:654] T 37abc6a50e6249c5a68fd3ddd9020f3a P b3a2757b35924642b12335f33832df9d: Neither blocks nor log segments found. Creating new log.
I20260619 10:17:12.876542 3051 tablet_bootstrap.cc:654] T 37abc6a50e6249c5a68fd3ddd9020f3a P 2b2d8effb39f4473b8bb107821ef1c81: Neither blocks nor log segments found. Creating new log.
I20260619 10:17:12.876991 2825 tablet_service.cc:1467] Processing CreateTablet for tablet 37abc6a50e6249c5a68fd3ddd9020f3a (DEFAULT_TABLE table=test-table [id=45aed87beac242acbf0f7f68c3703723]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 10:17:12.877101 2825 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 37abc6a50e6249c5a68fd3ddd9020f3a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 10:17:12.878300 3055 tablet_bootstrap.cc:492] T 37abc6a50e6249c5a68fd3ddd9020f3a P d869af37b51740718e3ea96f0972bc81: Bootstrap starting.
I20260619 10:17:12.878589 3096 raft_consensus.cc:2229] T 7535de8b335d4947bcc11e3c6aa0d418 P 2b2d8effb39f4473b8bb107821ef1c81 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 10:17:12.878684 3096 raft_consensus.cc:2258] T 7535de8b335d4947bcc11e3c6aa0d418 P 2b2d8effb39f4473b8bb107821ef1c81 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 10:17:12.878729 3055 tablet_bootstrap.cc:654] T 37abc6a50e6249c5a68fd3ddd9020f3a P d869af37b51740718e3ea96f0972bc81: Neither blocks nor log segments found. Creating new log.
I20260619 10:17:12.878820 3097 tablet_replica.cc:330] T 7535de8b335d4947bcc11e3c6aa0d418 P b3a2757b35924642b12335f33832df9d: stopping tablet replica
I20260619 10:17:12.878943 3098 tablet_replica.cc:330] T 7535de8b335d4947bcc11e3c6aa0d418 P d869af37b51740718e3ea96f0972bc81: stopping tablet replica
I20260619 10:17:12.879060 3097 raft_consensus.cc:2229] T 7535de8b335d4947bcc11e3c6aa0d418 P b3a2757b35924642b12335f33832df9d [term 1 LEADER]: Raft consensus shutting down.
I20260619 10:17:12.879137 3096 ts_tablet_manager.cc:1905] T 7535de8b335d4947bcc11e3c6aa0d418 P 2b2d8effb39f4473b8bb107821ef1c81: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 10:17:12.879217 3098 raft_consensus.cc:2229] T 7535de8b335d4947bcc11e3c6aa0d418 P d869af37b51740718e3ea96f0972bc81 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 10:17:12.879374 3097 raft_consensus.cc:2258] T 7535de8b335d4947bcc11e3c6aa0d418 P b3a2757b35924642b12335f33832df9d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 10:17:12.879477 3098 raft_consensus.cc:2258] T 7535de8b335d4947bcc11e3c6aa0d418 P d869af37b51740718e3ea96f0972bc81 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 10:17:12.879913 3097 ts_tablet_manager.cc:1905] T 7535de8b335d4947bcc11e3c6aa0d418 P b3a2757b35924642b12335f33832df9d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 10:17:12.881314 3098 ts_tablet_manager.cc:1905] T 7535de8b335d4947bcc11e3c6aa0d418 P d869af37b51740718e3ea96f0972bc81: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 10:17:12.881572 3097 ts_tablet_manager.cc:1918] T 7535de8b335d4947bcc11e3c6aa0d418 P b3a2757b35924642b12335f33832df9d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260619 10:17:12.881680 3097 log.cc:1196] T 7535de8b335d4947bcc11e3c6aa0d418 P b3a2757b35924642b12335f33832df9d: Deleting WAL directory at /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781864224577058-825-0/minicluster-data/ts-0/wal/wals/7535de8b335d4947bcc11e3c6aa0d418
I20260619 10:17:12.881989 3097 ts_tablet_manager.cc:1939] T 7535de8b335d4947bcc11e3c6aa0d418 P b3a2757b35924642b12335f33832df9d: Deleting consensus metadata
I20260619 10:17:12.882442 2532 catalog_manager.cc:4841] TS b3a2757b35924642b12335f33832df9d (127.0.206.65:35095): tablet 7535de8b335d4947bcc11e3c6aa0d418 (table test-table [id=c72ae3e8e59c4e229a1bef42057fbb77]) successfully deleted
I20260619 10:17:12.882978 3096 ts_tablet_manager.cc:1918] T 7535de8b335d4947bcc11e3c6aa0d418 P 2b2d8effb39f4473b8bb107821ef1c81: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260619 10:17:12.883100 3096 log.cc:1196] T 7535de8b335d4947bcc11e3c6aa0d418 P 2b2d8effb39f4473b8bb107821ef1c81: Deleting WAL directory at /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781864224577058-825-0/minicluster-data/ts-2/wal/wals/7535de8b335d4947bcc11e3c6aa0d418
I20260619 10:17:12.883190 3098 ts_tablet_manager.cc:1918] T 7535de8b335d4947bcc11e3c6aa0d418 P d869af37b51740718e3ea96f0972bc81: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260619 10:17:12.883260 3098 log.cc:1196] T 7535de8b335d4947bcc11e3c6aa0d418 P d869af37b51740718e3ea96f0972bc81: Deleting WAL directory at /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781864224577058-825-0/minicluster-data/ts-1/wal/wals/7535de8b335d4947bcc11e3c6aa0d418
I20260619 10:17:12.883490 3098 ts_tablet_manager.cc:1939] T 7535de8b335d4947bcc11e3c6aa0d418 P d869af37b51740718e3ea96f0972bc81: Deleting consensus metadata
I20260619 10:17:12.883639 3096 ts_tablet_manager.cc:1939] T 7535de8b335d4947bcc11e3c6aa0d418 P 2b2d8effb39f4473b8bb107821ef1c81: Deleting consensus metadata
I20260619 10:17:12.883862 2532 catalog_manager.cc:4841] TS d869af37b51740718e3ea96f0972bc81 (127.0.206.66:41427): tablet 7535de8b335d4947bcc11e3c6aa0d418 (table test-table [id=c72ae3e8e59c4e229a1bef42057fbb77]) successfully deleted
I20260619 10:17:12.884193 2534 catalog_manager.cc:4841] TS 2b2d8effb39f4473b8bb107821ef1c81 (127.0.206.67:35437): tablet 7535de8b335d4947bcc11e3c6aa0d418 (table test-table [id=c72ae3e8e59c4e229a1bef42057fbb77]) successfully deleted
I20260619 10:17:12.886965 3052 tablet_bootstrap.cc:492] T 37abc6a50e6249c5a68fd3ddd9020f3a P b3a2757b35924642b12335f33832df9d: No bootstrap required, opened a new log
I20260619 10:17:12.886991 3051 tablet_bootstrap.cc:492] T 37abc6a50e6249c5a68fd3ddd9020f3a P 2b2d8effb39f4473b8bb107821ef1c81: No bootstrap required, opened a new log
I20260619 10:17:12.887010 3052 ts_tablet_manager.cc:1397] T 37abc6a50e6249c5a68fd3ddd9020f3a P b3a2757b35924642b12335f33832df9d: Time spent bootstrapping tablet: real 0.011s user 0.001s sys 0.000s
I20260619 10:17:12.887036 3051 ts_tablet_manager.cc:1397] T 37abc6a50e6249c5a68fd3ddd9020f3a P 2b2d8effb39f4473b8bb107821ef1c81: Time spent bootstrapping tablet: real 0.011s user 0.001s sys 0.000s
I20260619 10:17:12.887133 3055 tablet_bootstrap.cc:492] T 37abc6a50e6249c5a68fd3ddd9020f3a P d869af37b51740718e3ea96f0972bc81: No bootstrap required, opened a new log
I20260619 10:17:12.887166 3055 ts_tablet_manager.cc:1397] T 37abc6a50e6249c5a68fd3ddd9020f3a P d869af37b51740718e3ea96f0972bc81: Time spent bootstrapping tablet: real 0.009s user 0.001s sys 0.000s
I20260619 10:17:12.887145 3052 raft_consensus.cc:348] T 37abc6a50e6249c5a68fd3ddd9020f3a P b3a2757b35924642b12335f33832df9d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3a2757b35924642b12335f33832df9d" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 35095 } } peers { permanent_uuid: "d869af37b51740718e3ea96f0972bc81" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 41427 } } peers { permanent_uuid: "2b2d8effb39f4473b8bb107821ef1c81" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 35437 } }
I20260619 10:17:12.887205 3052 raft_consensus.cc:374] T 37abc6a50e6249c5a68fd3ddd9020f3a P b3a2757b35924642b12335f33832df9d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 10:17:12.887166 3051 raft_consensus.cc:348] T 37abc6a50e6249c5a68fd3ddd9020f3a P 2b2d8effb39f4473b8bb107821ef1c81 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3a2757b35924642b12335f33832df9d" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 35095 } } peers { permanent_uuid: "d869af37b51740718e3ea96f0972bc81" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 41427 } } peers { permanent_uuid: "2b2d8effb39f4473b8bb107821ef1c81" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 35437 } }
I20260619 10:17:12.887225 3052 raft_consensus.cc:729] T 37abc6a50e6249c5a68fd3ddd9020f3a P b3a2757b35924642b12335f33832df9d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b3a2757b35924642b12335f33832df9d, State: Initialized, Role: FOLLOWER
I20260619 10:17:12.887235 3051 raft_consensus.cc:374] T 37abc6a50e6249c5a68fd3ddd9020f3a P 2b2d8effb39f4473b8bb107821ef1c81 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 10:17:12.887256 3051 raft_consensus.cc:729] T 37abc6a50e6249c5a68fd3ddd9020f3a P 2b2d8effb39f4473b8bb107821ef1c81 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2b2d8effb39f4473b8bb107821ef1c81, State: Initialized, Role: FOLLOWER
I20260619 10:17:12.887271 3052 consensus_queue.cc:260] T 37abc6a50e6249c5a68fd3ddd9020f3a P b3a2757b35924642b12335f33832df9d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3a2757b35924642b12335f33832df9d" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 35095 } } peers { permanent_uuid: "d869af37b51740718e3ea96f0972bc81" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 41427 } } peers { permanent_uuid: "2b2d8effb39f4473b8bb107821ef1c81" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 35437 } }
I20260619 10:17:12.887293 3051 consensus_queue.cc:260] T 37abc6a50e6249c5a68fd3ddd9020f3a P 2b2d8effb39f4473b8bb107821ef1c81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3a2757b35924642b12335f33832df9d" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 35095 } } peers { permanent_uuid: "d869af37b51740718e3ea96f0972bc81" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 41427 } } peers { permanent_uuid: "2b2d8effb39f4473b8bb107821ef1c81" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 35437 } }
I20260619 10:17:12.887295 3055 raft_consensus.cc:348] T 37abc6a50e6249c5a68fd3ddd9020f3a P d869af37b51740718e3ea96f0972bc81 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3a2757b35924642b12335f33832df9d" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 35095 } } peers { permanent_uuid: "d869af37b51740718e3ea96f0972bc81" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 41427 } } peers { permanent_uuid: "2b2d8effb39f4473b8bb107821ef1c81" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 35437 } }
I20260619 10:17:12.887352 3055 raft_consensus.cc:374] T 37abc6a50e6249c5a68fd3ddd9020f3a P d869af37b51740718e3ea96f0972bc81 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 10:17:12.887370 3055 raft_consensus.cc:729] T 37abc6a50e6249c5a68fd3ddd9020f3a P d869af37b51740718e3ea96f0972bc81 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d869af37b51740718e3ea96f0972bc81, State: Initialized, Role: FOLLOWER
I20260619 10:17:12.887373 3051 ts_tablet_manager.cc:1428] T 37abc6a50e6249c5a68fd3ddd9020f3a P 2b2d8effb39f4473b8bb107821ef1c81: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260619 10:17:12.887375 3052 ts_tablet_manager.cc:1428] T 37abc6a50e6249c5a68fd3ddd9020f3a P b3a2757b35924642b12335f33832df9d: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260619 10:17:12.887410 3055 consensus_queue.cc:260] T 37abc6a50e6249c5a68fd3ddd9020f3a P d869af37b51740718e3ea96f0972bc81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3a2757b35924642b12335f33832df9d" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 35095 } } peers { permanent_uuid: "d869af37b51740718e3ea96f0972bc81" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 41427 } } peers { permanent_uuid: "2b2d8effb39f4473b8bb107821ef1c81" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 35437 } }
I20260619 10:17:12.887512 3055 ts_tablet_manager.cc:1428] T 37abc6a50e6249c5a68fd3ddd9020f3a P d869af37b51740718e3ea96f0972bc81: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
W20260619 10:17:12.928608 3032 tablet.cc:2369] T 37abc6a50e6249c5a68fd3ddd9020f3a P 2b2d8effb39f4473b8bb107821ef1c81: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260619 10:17:13.123260 2752 tablet.cc:2369] T 37abc6a50e6249c5a68fd3ddd9020f3a P b3a2757b35924642b12335f33832df9d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 10:17:13.188961 3073 raft_consensus.cc:482] T 37abc6a50e6249c5a68fd3ddd9020f3a P 2b2d8effb39f4473b8bb107821ef1c81 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 10:17:13.189051 3073 raft_consensus.cc:504] T 37abc6a50e6249c5a68fd3ddd9020f3a P 2b2d8effb39f4473b8bb107821ef1c81 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3a2757b35924642b12335f33832df9d" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 35095 } } peers { permanent_uuid: "d869af37b51740718e3ea96f0972bc81" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 41427 } } peers { permanent_uuid: "2b2d8effb39f4473b8bb107821ef1c81" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 35437 } }
I20260619 10:17:13.189221 3073 leader_election.cc:288] T 37abc6a50e6249c5a68fd3ddd9020f3a P 2b2d8effb39f4473b8bb107821ef1c81 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b3a2757b35924642b12335f33832df9d (127.0.206.65:35095), d869af37b51740718e3ea96f0972bc81 (127.0.206.66:41427)
I20260619 10:17:13.189481 2705 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "37abc6a50e6249c5a68fd3ddd9020f3a" candidate_uuid: "2b2d8effb39f4473b8bb107821ef1c81" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b3a2757b35924642b12335f33832df9d" is_pre_election: true
I20260619 10:17:13.189589 2705 raft_consensus.cc:2454] T 37abc6a50e6249c5a68fd3ddd9020f3a P b3a2757b35924642b12335f33832df9d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2b2d8effb39f4473b8bb107821ef1c81 in term 0.
I20260619 10:17:13.189527 2845 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "37abc6a50e6249c5a68fd3ddd9020f3a" candidate_uuid: "2b2d8effb39f4473b8bb107821ef1c81" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d869af37b51740718e3ea96f0972bc81" is_pre_election: true
I20260619 10:17:13.189628 2845 raft_consensus.cc:2454] T 37abc6a50e6249c5a68fd3ddd9020f3a P d869af37b51740718e3ea96f0972bc81 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2b2d8effb39f4473b8bb107821ef1c81 in term 0.
I20260619 10:17:13.189775 2916 leader_election.cc:302] T 37abc6a50e6249c5a68fd3ddd9020f3a P 2b2d8effb39f4473b8bb107821ef1c81 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2b2d8effb39f4473b8bb107821ef1c81, b3a2757b35924642b12335f33832df9d; no voters:
I20260619 10:17:13.189894 3073 raft_consensus.cc:2781] T 37abc6a50e6249c5a68fd3ddd9020f3a P 2b2d8effb39f4473b8bb107821ef1c81 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 10:17:13.189961 3073 raft_consensus.cc:482] T 37abc6a50e6249c5a68fd3ddd9020f3a P 2b2d8effb39f4473b8bb107821ef1c81 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 10:17:13.189992 3073 raft_consensus.cc:3037] T 37abc6a50e6249c5a68fd3ddd9020f3a P 2b2d8effb39f4473b8bb107821ef1c81 [term 0 FOLLOWER]: Advancing to term 1
I20260619 10:17:13.190721 3073 raft_consensus.cc:504] T 37abc6a50e6249c5a68fd3ddd9020f3a P 2b2d8effb39f4473b8bb107821ef1c81 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3a2757b35924642b12335f33832df9d" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 35095 } } peers { permanent_uuid: "d869af37b51740718e3ea96f0972bc81" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 41427 } } peers { permanent_uuid: "2b2d8effb39f4473b8bb107821ef1c81" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 35437 } }
I20260619 10:17:13.190845 3073 leader_election.cc:288] T 37abc6a50e6249c5a68fd3ddd9020f3a P 2b2d8effb39f4473b8bb107821ef1c81 [CANDIDATE]: Term 1 election: Requested vote from peers b3a2757b35924642b12335f33832df9d (127.0.206.65:35095), d869af37b51740718e3ea96f0972bc81 (127.0.206.66:41427)
I20260619 10:17:13.190992 2705 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "37abc6a50e6249c5a68fd3ddd9020f3a" candidate_uuid: "2b2d8effb39f4473b8bb107821ef1c81" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b3a2757b35924642b12335f33832df9d"
I20260619 10:17:13.191001 2845 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "37abc6a50e6249c5a68fd3ddd9020f3a" candidate_uuid: "2b2d8effb39f4473b8bb107821ef1c81" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d869af37b51740718e3ea96f0972bc81"
I20260619 10:17:13.191072 2845 raft_consensus.cc:3037] T 37abc6a50e6249c5a68fd3ddd9020f3a P d869af37b51740718e3ea96f0972bc81 [term 0 FOLLOWER]: Advancing to term 1
I20260619 10:17:13.191088 2705 raft_consensus.cc:3037] T 37abc6a50e6249c5a68fd3ddd9020f3a P b3a2757b35924642b12335f33832df9d [term 0 FOLLOWER]: Advancing to term 1
I20260619 10:17:13.191704 2845 raft_consensus.cc:2454] T 37abc6a50e6249c5a68fd3ddd9020f3a P d869af37b51740718e3ea96f0972bc81 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2b2d8effb39f4473b8bb107821ef1c81 in term 1.
I20260619 10:17:13.191728 2705 raft_consensus.cc:2454] T 37abc6a50e6249c5a68fd3ddd9020f3a P b3a2757b35924642b12335f33832df9d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2b2d8effb39f4473b8bb107821ef1c81 in term 1.
I20260619 10:17:13.191895 2916 leader_election.cc:302] T 37abc6a50e6249c5a68fd3ddd9020f3a P 2b2d8effb39f4473b8bb107821ef1c81 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2b2d8effb39f4473b8bb107821ef1c81, b3a2757b35924642b12335f33832df9d; no voters:
I20260619 10:17:13.191998 3073 raft_consensus.cc:2781] T 37abc6a50e6249c5a68fd3ddd9020f3a P 2b2d8effb39f4473b8bb107821ef1c81 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 10:17:13.192059 3073 raft_consensus.cc:686] T 37abc6a50e6249c5a68fd3ddd9020f3a P 2b2d8effb39f4473b8bb107821ef1c81 [term 1 LEADER]: Becoming Leader. State: Replica: 2b2d8effb39f4473b8bb107821ef1c81, State: Running, Role: LEADER
I20260619 10:17:13.192109 3073 consensus_queue.cc:237] T 37abc6a50e6249c5a68fd3ddd9020f3a P 2b2d8effb39f4473b8bb107821ef1c81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3a2757b35924642b12335f33832df9d" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 35095 } } peers { permanent_uuid: "d869af37b51740718e3ea96f0972bc81" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 41427 } } peers { permanent_uuid: "2b2d8effb39f4473b8bb107821ef1c81" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 35437 } }
I20260619 10:17:13.192756 2547 catalog_manager.cc:5495] T 37abc6a50e6249c5a68fd3ddd9020f3a P 2b2d8effb39f4473b8bb107821ef1c81 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2b2d8effb39f4473b8bb107821ef1c81 (127.0.206.67). New cstate: current_term: 1 leader_uuid: "2b2d8effb39f4473b8bb107821ef1c81" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3a2757b35924642b12335f33832df9d" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 35095 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d869af37b51740718e3ea96f0972bc81" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 41427 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2b2d8effb39f4473b8bb107821ef1c81" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 35437 } health_report { overall_health: HEALTHY } } }
I20260619 10:17:13.234395 3048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:13.232559 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48644 (local address 127.0.206.67:35437)
0619 10:17:13.232597 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0619 10:17:13.232599 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 10:17:13.232630 (+ 31us) server_negotiation.cc:407] Connection header received
0619 10:17:13.232686 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:17:13.232687 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:17:13.232697 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:17:13.232737 (+ 40us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 10:17:13.232822 (+ 85us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:13.233254 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:13.233889 (+ 635us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:13.234052 (+ 163us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:13.234098 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:13.234220 (+ 122us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 10:17:13.234241 (+ 21us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 10:17:13.234271 (+ 30us) server_negotiation.cc:1035] Waiting for connection context
0619 10:17:13.234323 (+ 52us) server_negotiation.cc:299] Negotiation successful
0619 10:17:13.234338 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260619 10:17:13.238236 3048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:13.236302 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.67:55499 (local address 127.0.206.67:35437)
0619 10:17:13.236335 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0619 10:17:13.236337 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 10:17:13.236380 (+ 43us) server_negotiation.cc:407] Connection header received
0619 10:17:13.236439 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:17:13.236440 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:17:13.236448 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:17:13.236497 (+ 49us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 10:17:13.236605 (+ 108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:13.237031 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:13.237669 (+ 638us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:13.237843 (+ 174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:13.237911 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:13.238008 (+ 97us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 10:17:13.238034 (+ 26us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 10:17:13.238093 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0619 10:17:13.238138 (+ 45us) server_negotiation.cc:299] Negotiation successful
0619 10:17:13.238156 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":2,"server-negotiator.queue_time_us":18}
I20260619 10:17:13.238243 3037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:13.236282 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.67:35437 (local address 127.0.206.67:55499)
0619 10:17:13.236333 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0619 10:17:13.236344 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 10:17:13.236393 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 10:17:13.236500 (+ 107us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 10:17:13.236500 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 10:17:13.236501 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0619 10:17:13.236531 (+ 30us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:13.236532 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:13.237103 (+ 571us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:13.237104 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:13.237592 (+ 488us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:13.237593 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:13.237911 (+ 318us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:13.237911 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:13.237956 (+ 45us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:13.237959 (+ 3us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0619 10:17:13.238075 (+ 116us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0619 10:17:13.238093 (+ 18us) client_negotiation.cc:769] Sending connection context
0619 10:17:13.238136 (+ 43us) client_negotiation.cc:240] Negotiation successful
0619 10:17:13.238176 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34,"mutex_wait_us":2}
I20260619 10:17:13.240056 2845 raft_consensus.cc:1261] T 37abc6a50e6249c5a68fd3ddd9020f3a P d869af37b51740718e3ea96f0972bc81 [term 1 FOLLOWER]: Refusing update from remote peer 2b2d8effb39f4473b8bb107821ef1c81: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 10:17:13.240079 2705 raft_consensus.cc:1261] T 37abc6a50e6249c5a68fd3ddd9020f3a P b3a2757b35924642b12335f33832df9d [term 1 FOLLOWER]: Refusing update from remote peer 2b2d8effb39f4473b8bb107821ef1c81: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 10:17:13.240303 3056 consensus_queue.cc:1035] T 37abc6a50e6249c5a68fd3ddd9020f3a P 2b2d8effb39f4473b8bb107821ef1c81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d869af37b51740718e3ea96f0972bc81" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 41427 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 10:17:13.240411 3084 consensus_queue.cc:1035] T 37abc6a50e6249c5a68fd3ddd9020f3a P 2b2d8effb39f4473b8bb107821ef1c81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b3a2757b35924642b12335f33832df9d" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 35095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260619 10:17:13.244459 3078 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260619 10:17:14.761209 2548 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36476:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 10:17:14.761303 2548 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36476:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 10:17:14.762111 2548 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 28ae00894a9046c291da0695c210585d: Sending DeleteTablet for 3 replicas of tablet 37abc6a50e6249c5a68fd3ddd9020f3a
I20260619 10:17:14.762354 2685 tablet_service.cc:1514] Processing DeleteTablet for tablet 37abc6a50e6249c5a68fd3ddd9020f3a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 10:17:14 UTC) from {username='kudu', principal='kudu/127.0.206.126@KRBTEST.COM'} at 127.0.0.1:57326
I20260619 10:17:14.762414 2965 tablet_service.cc:1514] Processing DeleteTablet for tablet 37abc6a50e6249c5a68fd3ddd9020f3a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 10:17:14 UTC) from {username='kudu', principal='kudu/127.0.206.126@KRBTEST.COM'} at 127.0.0.1:48628
I20260619 10:17:14.762542 2825 tablet_service.cc:1514] Processing DeleteTablet for tablet 37abc6a50e6249c5a68fd3ddd9020f3a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 10:17:14 UTC) from {username='kudu', principal='kudu/127.0.206.126@KRBTEST.COM'} at 127.0.0.1:51598
I20260619 10:17:14.764241 3128 tablet_replica.cc:330] T 37abc6a50e6249c5a68fd3ddd9020f3a P d869af37b51740718e3ea96f0972bc81: stopping tablet replica
I20260619 10:17:14.764317 3128 raft_consensus.cc:2229] T 37abc6a50e6249c5a68fd3ddd9020f3a P d869af37b51740718e3ea96f0972bc81 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 10:17:14.764354 3128 raft_consensus.cc:2258] T 37abc6a50e6249c5a68fd3ddd9020f3a P d869af37b51740718e3ea96f0972bc81 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 10:17:14.764446 3127 tablet_replica.cc:330] T 37abc6a50e6249c5a68fd3ddd9020f3a P 2b2d8effb39f4473b8bb107821ef1c81: stopping tablet replica
I20260619 10:17:14.764503 3127 raft_consensus.cc:2229] T 37abc6a50e6249c5a68fd3ddd9020f3a P 2b2d8effb39f4473b8bb107821ef1c81 [term 1 LEADER]: Raft consensus shutting down.
I20260619 10:17:14.764583 3127 raft_consensus.cc:2258] T 37abc6a50e6249c5a68fd3ddd9020f3a P 2b2d8effb39f4473b8bb107821ef1c81 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 10:17:14.764868 3128 ts_tablet_manager.cc:1905] T 37abc6a50e6249c5a68fd3ddd9020f3a P d869af37b51740718e3ea96f0972bc81: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 10:17:14.766113 3126 tablet_replica.cc:330] T 37abc6a50e6249c5a68fd3ddd9020f3a P b3a2757b35924642b12335f33832df9d: stopping tablet replica
I20260619 10:17:14.766191 3126 raft_consensus.cc:2229] T 37abc6a50e6249c5a68fd3ddd9020f3a P b3a2757b35924642b12335f33832df9d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 10:17:14.766245 3126 raft_consensus.cc:2258] T 37abc6a50e6249c5a68fd3ddd9020f3a P b3a2757b35924642b12335f33832df9d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 10:17:14.766424 3127 ts_tablet_manager.cc:1905] T 37abc6a50e6249c5a68fd3ddd9020f3a P 2b2d8effb39f4473b8bb107821ef1c81: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 10:17:14.766562 3126 ts_tablet_manager.cc:1905] T 37abc6a50e6249c5a68fd3ddd9020f3a P b3a2757b35924642b12335f33832df9d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 10:17:14.767930 3126 ts_tablet_manager.cc:1918] T 37abc6a50e6249c5a68fd3ddd9020f3a P b3a2757b35924642b12335f33832df9d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260619 10:17:14.768009 3126 log.cc:1196] T 37abc6a50e6249c5a68fd3ddd9020f3a P b3a2757b35924642b12335f33832df9d: Deleting WAL directory at /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781864224577058-825-0/minicluster-data/ts-0/wal/wals/37abc6a50e6249c5a68fd3ddd9020f3a
I20260619 10:17:14.768107 3127 ts_tablet_manager.cc:1918] T 37abc6a50e6249c5a68fd3ddd9020f3a P 2b2d8effb39f4473b8bb107821ef1c81: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260619 10:17:14.768170 3127 log.cc:1196] T 37abc6a50e6249c5a68fd3ddd9020f3a P 2b2d8effb39f4473b8bb107821ef1c81: Deleting WAL directory at /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781864224577058-825-0/minicluster-data/ts-2/wal/wals/37abc6a50e6249c5a68fd3ddd9020f3a
I20260619 10:17:14.768265 3126 ts_tablet_manager.cc:1939] T 37abc6a50e6249c5a68fd3ddd9020f3a P b3a2757b35924642b12335f33832df9d: Deleting consensus metadata
I20260619 10:17:14.768414 3127 ts_tablet_manager.cc:1939] T 37abc6a50e6249c5a68fd3ddd9020f3a P 2b2d8effb39f4473b8bb107821ef1c81: Deleting consensus metadata
I20260619 10:17:14.768648 2532 catalog_manager.cc:4841] TS b3a2757b35924642b12335f33832df9d (127.0.206.65:35095): tablet 37abc6a50e6249c5a68fd3ddd9020f3a (table test-table [id=45aed87beac242acbf0f7f68c3703723]) successfully deleted
I20260619 10:17:14.768743 3128 ts_tablet_manager.cc:1918] T 37abc6a50e6249c5a68fd3ddd9020f3a P d869af37b51740718e3ea96f0972bc81: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260619 10:17:14.768757 2534 catalog_manager.cc:4841] TS 2b2d8effb39f4473b8bb107821ef1c81 (127.0.206.67:35437): tablet 37abc6a50e6249c5a68fd3ddd9020f3a (table test-table [id=45aed87beac242acbf0f7f68c3703723]) successfully deleted
I20260619 10:17:14.768796 3128 log.cc:1196] T 37abc6a50e6249c5a68fd3ddd9020f3a P d869af37b51740718e3ea96f0972bc81: Deleting WAL directory at /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781864224577058-825-0/minicluster-data/ts-1/wal/wals/37abc6a50e6249c5a68fd3ddd9020f3a
I20260619 10:17:14.769080 3128 ts_tablet_manager.cc:1939] T 37abc6a50e6249c5a68fd3ddd9020f3a P d869af37b51740718e3ea96f0972bc81: Deleting consensus metadata
I20260619 10:17:14.769397 2532 catalog_manager.cc:4841] TS d869af37b51740718e3ea96f0972bc81 (127.0.206.66:41427): tablet 37abc6a50e6249c5a68fd3ddd9020f3a (table test-table [id=45aed87beac242acbf0f7f68c3703723]) successfully deleted
Jun 19 10:17:14 dist-test-slave-6pt2 krb5kdc[2503](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864234, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Jun 19 10:17:14 dist-test-slave-6pt2 krb5kdc[2503](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864234, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.0.206.126@KRBTEST.COM
I20260619 10:17:14.786234 3136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:14.778526 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36498 (local address 127.0.206.126:36165)
0619 10:17:14.778708 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0619 10:17:14.778717 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 10:17:14.778726 (+ 9us) server_negotiation.cc:407] Connection header received
0619 10:17:14.778750 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:17:14.778751 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:17:14.778761 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:17:14.778844 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:17:14.779410 (+ 566us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:14.779921 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:14.780746 (+ 825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:14.780960 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:14.781033 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:14.783686 (+ 2653us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 10:17:14.783687 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 10:17:14.783688 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 10:17:14.783694 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 10:17:14.785110 (+ 1416us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:17:14.785494 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:17:14.785495 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:17:14.785495 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:17:14.785542 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:17:14.785761 (+ 219us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:17:14.785762 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:17:14.785762 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:17:14.785864 (+ 102us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 10:17:14.785949 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0619 10:17:14.786095 (+ 146us) server_negotiation.cc:299] Negotiation successful
0619 10:17:14.786156 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":151,"thread_start_us":109,"threads_started":1}
I20260619 10:17:14.787938 2548 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:36498:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 10:17:14.788103 2548 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 10:17:14.789451 2685 tablet_service.cc:1467] Processing CreateTablet for tablet 29d43e5bd7ec4ebfbefe2df074d23617 (DEFAULT_TABLE table=test-table [id=9646b60dda894e21b62ce9f1fa185310]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 10:17:14.789465 2965 tablet_service.cc:1467] Processing CreateTablet for tablet 29d43e5bd7ec4ebfbefe2df074d23617 (DEFAULT_TABLE table=test-table [id=9646b60dda894e21b62ce9f1fa185310]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 10:17:14.789593 2685 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 29d43e5bd7ec4ebfbefe2df074d23617. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 10:17:14.789599 2965 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 29d43e5bd7ec4ebfbefe2df074d23617. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 10:17:14.790670 3137 tablet_bootstrap.cc:492] T 29d43e5bd7ec4ebfbefe2df074d23617 P 2b2d8effb39f4473b8bb107821ef1c81: Bootstrap starting.
I20260619 10:17:14.790675 3138 tablet_bootstrap.cc:492] T 29d43e5bd7ec4ebfbefe2df074d23617 P b3a2757b35924642b12335f33832df9d: Bootstrap starting.
I20260619 10:17:14.790729 2825 tablet_service.cc:1467] Processing CreateTablet for tablet 29d43e5bd7ec4ebfbefe2df074d23617 (DEFAULT_TABLE table=test-table [id=9646b60dda894e21b62ce9f1fa185310]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 10:17:14.790849 2825 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 29d43e5bd7ec4ebfbefe2df074d23617. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 10:17:14.790985 3138 tablet_bootstrap.cc:654] T 29d43e5bd7ec4ebfbefe2df074d23617 P b3a2757b35924642b12335f33832df9d: Neither blocks nor log segments found. Creating new log.
I20260619 10:17:14.791025 3137 tablet_bootstrap.cc:654] T 29d43e5bd7ec4ebfbefe2df074d23617 P 2b2d8effb39f4473b8bb107821ef1c81: Neither blocks nor log segments found. Creating new log.
I20260619 10:17:14.791568 3138 tablet_bootstrap.cc:492] T 29d43e5bd7ec4ebfbefe2df074d23617 P b3a2757b35924642b12335f33832df9d: No bootstrap required, opened a new log
I20260619 10:17:14.791632 3138 ts_tablet_manager.cc:1397] T 29d43e5bd7ec4ebfbefe2df074d23617 P b3a2757b35924642b12335f33832df9d: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260619 10:17:14.791745 3138 raft_consensus.cc:348] T 29d43e5bd7ec4ebfbefe2df074d23617 P b3a2757b35924642b12335f33832df9d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3a2757b35924642b12335f33832df9d" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 35095 } } peers { permanent_uuid: "d869af37b51740718e3ea96f0972bc81" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 41427 } } peers { permanent_uuid: "2b2d8effb39f4473b8bb107821ef1c81" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 35437 } }
I20260619 10:17:14.791817 3138 raft_consensus.cc:374] T 29d43e5bd7ec4ebfbefe2df074d23617 P b3a2757b35924642b12335f33832df9d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 10:17:14.791848 3138 raft_consensus.cc:729] T 29d43e5bd7ec4ebfbefe2df074d23617 P b3a2757b35924642b12335f33832df9d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b3a2757b35924642b12335f33832df9d, State: Initialized, Role: FOLLOWER
I20260619 10:17:14.791901 3138 consensus_queue.cc:260] T 29d43e5bd7ec4ebfbefe2df074d23617 P b3a2757b35924642b12335f33832df9d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3a2757b35924642b12335f33832df9d" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 35095 } } peers { permanent_uuid: "d869af37b51740718e3ea96f0972bc81" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 41427 } } peers { permanent_uuid: "2b2d8effb39f4473b8bb107821ef1c81" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 35437 } }
I20260619 10:17:14.792058 3141 tablet_bootstrap.cc:492] T 29d43e5bd7ec4ebfbefe2df074d23617 P d869af37b51740718e3ea96f0972bc81: Bootstrap starting.
I20260619 10:17:14.792064 3138 ts_tablet_manager.cc:1428] T 29d43e5bd7ec4ebfbefe2df074d23617 P b3a2757b35924642b12335f33832df9d: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260619 10:17:14.792380 3141 tablet_bootstrap.cc:654] T 29d43e5bd7ec4ebfbefe2df074d23617 P d869af37b51740718e3ea96f0972bc81: Neither blocks nor log segments found. Creating new log.
I20260619 10:17:14.792583 3137 tablet_bootstrap.cc:492] T 29d43e5bd7ec4ebfbefe2df074d23617 P 2b2d8effb39f4473b8bb107821ef1c81: No bootstrap required, opened a new log
I20260619 10:17:14.792657 3137 ts_tablet_manager.cc:1397] T 29d43e5bd7ec4ebfbefe2df074d23617 P 2b2d8effb39f4473b8bb107821ef1c81: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260619 10:17:14.792780 3137 raft_consensus.cc:348] T 29d43e5bd7ec4ebfbefe2df074d23617 P 2b2d8effb39f4473b8bb107821ef1c81 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3a2757b35924642b12335f33832df9d" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 35095 } } peers { permanent_uuid: "d869af37b51740718e3ea96f0972bc81" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 41427 } } peers { permanent_uuid: "2b2d8effb39f4473b8bb107821ef1c81" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 35437 } }
I20260619 10:17:14.792855 3137 raft_consensus.cc:374] T 29d43e5bd7ec4ebfbefe2df074d23617 P 2b2d8effb39f4473b8bb107821ef1c81 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 10:17:14.792874 3137 raft_consensus.cc:729] T 29d43e5bd7ec4ebfbefe2df074d23617 P 2b2d8effb39f4473b8bb107821ef1c81 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2b2d8effb39f4473b8bb107821ef1c81, State: Initialized, Role: FOLLOWER
I20260619 10:17:14.792917 3137 consensus_queue.cc:260] T 29d43e5bd7ec4ebfbefe2df074d23617 P 2b2d8effb39f4473b8bb107821ef1c81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3a2757b35924642b12335f33832df9d" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 35095 } } peers { permanent_uuid: "d869af37b51740718e3ea96f0972bc81" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 41427 } } peers { permanent_uuid: "2b2d8effb39f4473b8bb107821ef1c81" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 35437 } }
I20260619 10:17:14.792975 3141 tablet_bootstrap.cc:492] T 29d43e5bd7ec4ebfbefe2df074d23617 P d869af37b51740718e3ea96f0972bc81: No bootstrap required, opened a new log
I20260619 10:17:14.793007 3137 ts_tablet_manager.cc:1428] T 29d43e5bd7ec4ebfbefe2df074d23617 P 2b2d8effb39f4473b8bb107821ef1c81: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260619 10:17:14.793018 3141 ts_tablet_manager.cc:1397] T 29d43e5bd7ec4ebfbefe2df074d23617 P d869af37b51740718e3ea96f0972bc81: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260619 10:17:14.793146 3141 raft_consensus.cc:348] T 29d43e5bd7ec4ebfbefe2df074d23617 P d869af37b51740718e3ea96f0972bc81 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3a2757b35924642b12335f33832df9d" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 35095 } } peers { permanent_uuid: "d869af37b51740718e3ea96f0972bc81" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 41427 } } peers { permanent_uuid: "2b2d8effb39f4473b8bb107821ef1c81" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 35437 } }
I20260619 10:17:14.793215 3141 raft_consensus.cc:374] T 29d43e5bd7ec4ebfbefe2df074d23617 P d869af37b51740718e3ea96f0972bc81 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 10:17:14.793233 3141 raft_consensus.cc:729] T 29d43e5bd7ec4ebfbefe2df074d23617 P d869af37b51740718e3ea96f0972bc81 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d869af37b51740718e3ea96f0972bc81, State: Initialized, Role: FOLLOWER
I20260619 10:17:14.793272 3141 consensus_queue.cc:260] T 29d43e5bd7ec4ebfbefe2df074d23617 P d869af37b51740718e3ea96f0972bc81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3a2757b35924642b12335f33832df9d" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 35095 } } peers { permanent_uuid: "d869af37b51740718e3ea96f0972bc81" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 41427 } } peers { permanent_uuid: "2b2d8effb39f4473b8bb107821ef1c81" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 35437 } }
I20260619 10:17:14.793409 3141 ts_tablet_manager.cc:1428] T 29d43e5bd7ec4ebfbefe2df074d23617 P d869af37b51740718e3ea96f0972bc81: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260619 10:17:14.864351 3114 raft_consensus.cc:482] T 29d43e5bd7ec4ebfbefe2df074d23617 P 2b2d8effb39f4473b8bb107821ef1c81 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 10:17:14.864475 3114 raft_consensus.cc:504] T 29d43e5bd7ec4ebfbefe2df074d23617 P 2b2d8effb39f4473b8bb107821ef1c81 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3a2757b35924642b12335f33832df9d" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 35095 } } peers { permanent_uuid: "d869af37b51740718e3ea96f0972bc81" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 41427 } } peers { permanent_uuid: "2b2d8effb39f4473b8bb107821ef1c81" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 35437 } }
I20260619 10:17:14.864675 3114 leader_election.cc:288] T 29d43e5bd7ec4ebfbefe2df074d23617 P 2b2d8effb39f4473b8bb107821ef1c81 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b3a2757b35924642b12335f33832df9d (127.0.206.65:35095), d869af37b51740718e3ea96f0972bc81 (127.0.206.66:41427)
I20260619 10:17:14.864940 2704 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "29d43e5bd7ec4ebfbefe2df074d23617" candidate_uuid: "2b2d8effb39f4473b8bb107821ef1c81" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b3a2757b35924642b12335f33832df9d" is_pre_election: true
I20260619 10:17:14.864979 2844 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "29d43e5bd7ec4ebfbefe2df074d23617" candidate_uuid: "2b2d8effb39f4473b8bb107821ef1c81" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d869af37b51740718e3ea96f0972bc81" is_pre_election: true
I20260619 10:17:14.865048 2704 raft_consensus.cc:2454] T 29d43e5bd7ec4ebfbefe2df074d23617 P b3a2757b35924642b12335f33832df9d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2b2d8effb39f4473b8bb107821ef1c81 in term 0.
I20260619 10:17:14.865057 2844 raft_consensus.cc:2454] T 29d43e5bd7ec4ebfbefe2df074d23617 P d869af37b51740718e3ea96f0972bc81 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2b2d8effb39f4473b8bb107821ef1c81 in term 0.
I20260619 10:17:14.865192 2916 leader_election.cc:302] T 29d43e5bd7ec4ebfbefe2df074d23617 P 2b2d8effb39f4473b8bb107821ef1c81 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2b2d8effb39f4473b8bb107821ef1c81, b3a2757b35924642b12335f33832df9d; no voters:
I20260619 10:17:14.865303 3114 raft_consensus.cc:2781] T 29d43e5bd7ec4ebfbefe2df074d23617 P 2b2d8effb39f4473b8bb107821ef1c81 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 10:17:14.865341 3114 raft_consensus.cc:482] T 29d43e5bd7ec4ebfbefe2df074d23617 P 2b2d8effb39f4473b8bb107821ef1c81 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 10:17:14.865367 3114 raft_consensus.cc:3037] T 29d43e5bd7ec4ebfbefe2df074d23617 P 2b2d8effb39f4473b8bb107821ef1c81 [term 0 FOLLOWER]: Advancing to term 1
I20260619 10:17:14.866032 3114 raft_consensus.cc:504] T 29d43e5bd7ec4ebfbefe2df074d23617 P 2b2d8effb39f4473b8bb107821ef1c81 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3a2757b35924642b12335f33832df9d" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 35095 } } peers { permanent_uuid: "d869af37b51740718e3ea96f0972bc81" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 41427 } } peers { permanent_uuid: "2b2d8effb39f4473b8bb107821ef1c81" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 35437 } }
I20260619 10:17:14.866154 3114 leader_election.cc:288] T 29d43e5bd7ec4ebfbefe2df074d23617 P 2b2d8effb39f4473b8bb107821ef1c81 [CANDIDATE]: Term 1 election: Requested vote from peers b3a2757b35924642b12335f33832df9d (127.0.206.65:35095), d869af37b51740718e3ea96f0972bc81 (127.0.206.66:41427)
I20260619 10:17:14.866273 2704 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "29d43e5bd7ec4ebfbefe2df074d23617" candidate_uuid: "2b2d8effb39f4473b8bb107821ef1c81" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b3a2757b35924642b12335f33832df9d"
I20260619 10:17:14.866374 2704 raft_consensus.cc:3037] T 29d43e5bd7ec4ebfbefe2df074d23617 P b3a2757b35924642b12335f33832df9d [term 0 FOLLOWER]: Advancing to term 1
I20260619 10:17:14.866322 2844 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "29d43e5bd7ec4ebfbefe2df074d23617" candidate_uuid: "2b2d8effb39f4473b8bb107821ef1c81" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d869af37b51740718e3ea96f0972bc81"
I20260619 10:17:14.866417 2844 raft_consensus.cc:3037] T 29d43e5bd7ec4ebfbefe2df074d23617 P d869af37b51740718e3ea96f0972bc81 [term 0 FOLLOWER]: Advancing to term 1
I20260619 10:17:14.867080 2844 raft_consensus.cc:2454] T 29d43e5bd7ec4ebfbefe2df074d23617 P d869af37b51740718e3ea96f0972bc81 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2b2d8effb39f4473b8bb107821ef1c81 in term 1.
I20260619 10:17:14.867089 2704 raft_consensus.cc:2454] T 29d43e5bd7ec4ebfbefe2df074d23617 P b3a2757b35924642b12335f33832df9d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2b2d8effb39f4473b8bb107821ef1c81 in term 1.
I20260619 10:17:14.867242 2916 leader_election.cc:302] T 29d43e5bd7ec4ebfbefe2df074d23617 P 2b2d8effb39f4473b8bb107821ef1c81 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2b2d8effb39f4473b8bb107821ef1c81, b3a2757b35924642b12335f33832df9d; no voters:
I20260619 10:17:14.867374 3114 raft_consensus.cc:2781] T 29d43e5bd7ec4ebfbefe2df074d23617 P 2b2d8effb39f4473b8bb107821ef1c81 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 10:17:14.867431 3114 raft_consensus.cc:686] T 29d43e5bd7ec4ebfbefe2df074d23617 P 2b2d8effb39f4473b8bb107821ef1c81 [term 1 LEADER]: Becoming Leader. State: Replica: 2b2d8effb39f4473b8bb107821ef1c81, State: Running, Role: LEADER
I20260619 10:17:14.867499 3114 consensus_queue.cc:237] T 29d43e5bd7ec4ebfbefe2df074d23617 P 2b2d8effb39f4473b8bb107821ef1c81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3a2757b35924642b12335f33832df9d" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 35095 } } peers { permanent_uuid: "d869af37b51740718e3ea96f0972bc81" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 41427 } } peers { permanent_uuid: "2b2d8effb39f4473b8bb107821ef1c81" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 35437 } }
I20260619 10:17:14.868067 2548 catalog_manager.cc:5495] T 29d43e5bd7ec4ebfbefe2df074d23617 P 2b2d8effb39f4473b8bb107821ef1c81 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2b2d8effb39f4473b8bb107821ef1c81 (127.0.206.67). New cstate: current_term: 1 leader_uuid: "2b2d8effb39f4473b8bb107821ef1c81" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3a2757b35924642b12335f33832df9d" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 35095 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d869af37b51740718e3ea96f0972bc81" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 41427 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2b2d8effb39f4473b8bb107821ef1c81" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 35437 } health_report { overall_health: HEALTHY } } }
I20260619 10:17:14.888455 3147 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:14.886210 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48656 (local address 127.0.206.67:35437)
0619 10:17:14.886364 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0619 10:17:14.886366 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 10:17:14.886389 (+ 23us) server_negotiation.cc:407] Connection header received
0619 10:17:14.886408 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:17:14.886409 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:17:14.886423 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:17:14.886488 (+ 65us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 10:17:14.886588 (+ 100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:14.887041 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:14.887791 (+ 750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:14.888003 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:14.888070 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:14.888209 (+ 139us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 10:17:14.888238 (+ 29us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 10:17:14.888280 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0619 10:17:14.888342 (+ 62us) server_negotiation.cc:299] Negotiation successful
0619 10:17:14.888360 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":131,"thread_start_us":93,"threads_started":1}
I20260619 10:17:14.890298 2844 raft_consensus.cc:1261] T 29d43e5bd7ec4ebfbefe2df074d23617 P d869af37b51740718e3ea96f0972bc81 [term 1 FOLLOWER]: Refusing update from remote peer 2b2d8effb39f4473b8bb107821ef1c81: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 10:17:14.890432 2704 raft_consensus.cc:1261] T 29d43e5bd7ec4ebfbefe2df074d23617 P b3a2757b35924642b12335f33832df9d [term 1 FOLLOWER]: Refusing update from remote peer 2b2d8effb39f4473b8bb107821ef1c81: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 10:17:14.890492 3114 consensus_queue.cc:1035] T 29d43e5bd7ec4ebfbefe2df074d23617 P 2b2d8effb39f4473b8bb107821ef1c81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d869af37b51740718e3ea96f0972bc81" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 41427 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 10:17:14.890609 3109 consensus_queue.cc:1035] T 29d43e5bd7ec4ebfbefe2df074d23617 P 2b2d8effb39f4473b8bb107821ef1c81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b3a2757b35924642b12335f33832df9d" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 35095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 10:17:14.899884 2547 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:36498:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 10:17:14.899960 2547 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:36498:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 10:17:14.900513 2547 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 28ae00894a9046c291da0695c210585d: Sending DeleteTablet for 3 replicas of tablet 29d43e5bd7ec4ebfbefe2df074d23617
I20260619 10:17:14.900712 2685 tablet_service.cc:1514] Processing DeleteTablet for tablet 29d43e5bd7ec4ebfbefe2df074d23617 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 10:17:14 UTC) from {username='kudu', principal='kudu/127.0.206.126@KRBTEST.COM'} at 127.0.0.1:57326
I20260619 10:17:14.900738 2964 tablet_service.cc:1514] Processing DeleteTablet for tablet 29d43e5bd7ec4ebfbefe2df074d23617 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 10:17:14 UTC) from {username='kudu', principal='kudu/127.0.206.126@KRBTEST.COM'} at 127.0.0.1:48628
I20260619 10:17:14.900821 3127 tablet_replica.cc:330] T 29d43e5bd7ec4ebfbefe2df074d23617 P 2b2d8effb39f4473b8bb107821ef1c81: stopping tablet replica
I20260619 10:17:14.900812 2825 tablet_service.cc:1514] Processing DeleteTablet for tablet 29d43e5bd7ec4ebfbefe2df074d23617 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 10:17:14 UTC) from {username='kudu', principal='kudu/127.0.206.126@KRBTEST.COM'} at 127.0.0.1:51598
I20260619 10:17:14.900874 3127 raft_consensus.cc:2229] T 29d43e5bd7ec4ebfbefe2df074d23617 P 2b2d8effb39f4473b8bb107821ef1c81 [term 1 LEADER]: Raft consensus shutting down.
I20260619 10:17:14.900890 3128 tablet_replica.cc:330] T 29d43e5bd7ec4ebfbefe2df074d23617 P d869af37b51740718e3ea96f0972bc81: stopping tablet replica
I20260619 10:17:14.900893 3126 tablet_replica.cc:330] T 29d43e5bd7ec4ebfbefe2df074d23617 P b3a2757b35924642b12335f33832df9d: stopping tablet replica
I20260619 10:17:14.900923 3127 raft_consensus.cc:2258] T 29d43e5bd7ec4ebfbefe2df074d23617 P 2b2d8effb39f4473b8bb107821ef1c81 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 10:17:14.900936 3128 raft_consensus.cc:2229] T 29d43e5bd7ec4ebfbefe2df074d23617 P d869af37b51740718e3ea96f0972bc81 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 10:17:14.900940 3126 raft_consensus.cc:2229] T 29d43e5bd7ec4ebfbefe2df074d23617 P b3a2757b35924642b12335f33832df9d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 10:17:14.900978 3128 raft_consensus.cc:2258] T 29d43e5bd7ec4ebfbefe2df074d23617 P d869af37b51740718e3ea96f0972bc81 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 10:17:14.900980 3126 raft_consensus.cc:2258] T 29d43e5bd7ec4ebfbefe2df074d23617 P b3a2757b35924642b12335f33832df9d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 10:17:14.901216 3127 ts_tablet_manager.cc:1905] T 29d43e5bd7ec4ebfbefe2df074d23617 P 2b2d8effb39f4473b8bb107821ef1c81: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 10:17:14.901180 2547 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:36498:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260619 10:17:14.901242 3128 ts_tablet_manager.cc:1905] T 29d43e5bd7ec4ebfbefe2df074d23617 P d869af37b51740718e3ea96f0972bc81: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 10:17:14.901252 3126 ts_tablet_manager.cc:1905] T 29d43e5bd7ec4ebfbefe2df074d23617 P b3a2757b35924642b12335f33832df9d: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260619 10:17:14.901307 2547 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 10:17:14.902832 3126 ts_tablet_manager.cc:1918] T 29d43e5bd7ec4ebfbefe2df074d23617 P b3a2757b35924642b12335f33832df9d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260619 10:17:14.902891 3126 log.cc:1196] T 29d43e5bd7ec4ebfbefe2df074d23617 P b3a2757b35924642b12335f33832df9d: Deleting WAL directory at /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781864224577058-825-0/minicluster-data/ts-0/wal/wals/29d43e5bd7ec4ebfbefe2df074d23617
I20260619 10:17:14.902869 3128 ts_tablet_manager.cc:1918] T 29d43e5bd7ec4ebfbefe2df074d23617 P d869af37b51740718e3ea96f0972bc81: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260619 10:17:14.902932 3128 log.cc:1196] T 29d43e5bd7ec4ebfbefe2df074d23617 P d869af37b51740718e3ea96f0972bc81: Deleting WAL directory at /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781864224577058-825-0/minicluster-data/ts-1/wal/wals/29d43e5bd7ec4ebfbefe2df074d23617
I20260619 10:17:14.903151 3126 ts_tablet_manager.cc:1939] T 29d43e5bd7ec4ebfbefe2df074d23617 P b3a2757b35924642b12335f33832df9d: Deleting consensus metadata
I20260619 10:17:14.903173 3128 ts_tablet_manager.cc:1939] T 29d43e5bd7ec4ebfbefe2df074d23617 P d869af37b51740718e3ea96f0972bc81: Deleting consensus metadata
I20260619 10:17:14.903487 2532 catalog_manager.cc:4841] TS b3a2757b35924642b12335f33832df9d (127.0.206.65:35095): tablet 29d43e5bd7ec4ebfbefe2df074d23617 (table test-table [id=9646b60dda894e21b62ce9f1fa185310]) successfully deleted
I20260619 10:17:14.903537 2964 tablet_service.cc:1467] Processing CreateTablet for tablet 6dcfa1d900634f87afde19e4dfa48a72 (DEFAULT_TABLE table=test-table [id=9a15f0f8f38842fa9cf2e274fe2d5c38]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 10:17:14.903563 2825 tablet_service.cc:1467] Processing CreateTablet for tablet 6dcfa1d900634f87afde19e4dfa48a72 (DEFAULT_TABLE table=test-table [id=9a15f0f8f38842fa9cf2e274fe2d5c38]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 10:17:14.903604 2532 catalog_manager.cc:4841] TS d869af37b51740718e3ea96f0972bc81 (127.0.206.66:41427): tablet 29d43e5bd7ec4ebfbefe2df074d23617 (table test-table [id=9646b60dda894e21b62ce9f1fa185310]) successfully deleted
I20260619 10:17:14.903663 2825 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6dcfa1d900634f87afde19e4dfa48a72. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 10:17:14.903752 2685 tablet_service.cc:1467] Processing CreateTablet for tablet 6dcfa1d900634f87afde19e4dfa48a72 (DEFAULT_TABLE table=test-table [id=9a15f0f8f38842fa9cf2e274fe2d5c38]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 10:17:14.903836 3127 ts_tablet_manager.cc:1918] T 29d43e5bd7ec4ebfbefe2df074d23617 P 2b2d8effb39f4473b8bb107821ef1c81: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260619 10:17:14.903848 2685 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6dcfa1d900634f87afde19e4dfa48a72. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 10:17:14.903888 3127 log.cc:1196] T 29d43e5bd7ec4ebfbefe2df074d23617 P 2b2d8effb39f4473b8bb107821ef1c81: Deleting WAL directory at /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781864224577058-825-0/minicluster-data/ts-2/wal/wals/29d43e5bd7ec4ebfbefe2df074d23617
I20260619 10:17:14.904093 3127 ts_tablet_manager.cc:1939] T 29d43e5bd7ec4ebfbefe2df074d23617 P 2b2d8effb39f4473b8bb107821ef1c81: Deleting consensus metadata
I20260619 10:17:14.904237 2964 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6dcfa1d900634f87afde19e4dfa48a72. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 10:17:14.904831 3141 tablet_bootstrap.cc:492] T 6dcfa1d900634f87afde19e4dfa48a72 P d869af37b51740718e3ea96f0972bc81: Bootstrap starting.
I20260619 10:17:14.904832 3138 tablet_bootstrap.cc:492] T 6dcfa1d900634f87afde19e4dfa48a72 P b3a2757b35924642b12335f33832df9d: Bootstrap starting.
I20260619 10:17:14.905007 2534 catalog_manager.cc:4841] TS 2b2d8effb39f4473b8bb107821ef1c81 (127.0.206.67:35437): tablet 29d43e5bd7ec4ebfbefe2df074d23617 (table test-table [id=9646b60dda894e21b62ce9f1fa185310]) successfully deleted
I20260619 10:17:14.905164 3138 tablet_bootstrap.cc:654] T 6dcfa1d900634f87afde19e4dfa48a72 P b3a2757b35924642b12335f33832df9d: Neither blocks nor log segments found. Creating new log.
I20260619 10:17:14.905198 3141 tablet_bootstrap.cc:654] T 6dcfa1d900634f87afde19e4dfa48a72 P d869af37b51740718e3ea96f0972bc81: Neither blocks nor log segments found. Creating new log.
I20260619 10:17:14.905555 3137 tablet_bootstrap.cc:492] T 6dcfa1d900634f87afde19e4dfa48a72 P 2b2d8effb39f4473b8bb107821ef1c81: Bootstrap starting.
I20260619 10:17:14.905858 3137 tablet_bootstrap.cc:654] T 6dcfa1d900634f87afde19e4dfa48a72 P 2b2d8effb39f4473b8bb107821ef1c81: Neither blocks nor log segments found. Creating new log.
I20260619 10:17:14.905989 3138 tablet_bootstrap.cc:492] T 6dcfa1d900634f87afde19e4dfa48a72 P b3a2757b35924642b12335f33832df9d: No bootstrap required, opened a new log
I20260619 10:17:14.906040 3138 ts_tablet_manager.cc:1397] T 6dcfa1d900634f87afde19e4dfa48a72 P b3a2757b35924642b12335f33832df9d: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260619 10:17:14.906170 3138 raft_consensus.cc:348] T 6dcfa1d900634f87afde19e4dfa48a72 P b3a2757b35924642b12335f33832df9d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3a2757b35924642b12335f33832df9d" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 35095 } } peers { permanent_uuid: "d869af37b51740718e3ea96f0972bc81" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 41427 } } peers { permanent_uuid: "2b2d8effb39f4473b8bb107821ef1c81" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 35437 } }
I20260619 10:17:14.906248 3138 raft_consensus.cc:374] T 6dcfa1d900634f87afde19e4dfa48a72 P b3a2757b35924642b12335f33832df9d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 10:17:14.906273 3138 raft_consensus.cc:729] T 6dcfa1d900634f87afde19e4dfa48a72 P b3a2757b35924642b12335f33832df9d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b3a2757b35924642b12335f33832df9d, State: Initialized, Role: FOLLOWER
I20260619 10:17:14.906328 3138 consensus_queue.cc:260] T 6dcfa1d900634f87afde19e4dfa48a72 P b3a2757b35924642b12335f33832df9d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3a2757b35924642b12335f33832df9d" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 35095 } } peers { permanent_uuid: "d869af37b51740718e3ea96f0972bc81" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 41427 } } peers { permanent_uuid: "2b2d8effb39f4473b8bb107821ef1c81" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 35437 } }
I20260619 10:17:14.906422 3138 ts_tablet_manager.cc:1428] T 6dcfa1d900634f87afde19e4dfa48a72 P b3a2757b35924642b12335f33832df9d: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260619 10:17:14.906495 3137 tablet_bootstrap.cc:492] T 6dcfa1d900634f87afde19e4dfa48a72 P 2b2d8effb39f4473b8bb107821ef1c81: No bootstrap required, opened a new log
I20260619 10:17:14.906539 3137 ts_tablet_manager.cc:1397] T 6dcfa1d900634f87afde19e4dfa48a72 P 2b2d8effb39f4473b8bb107821ef1c81: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260619 10:17:14.906656 3141 tablet_bootstrap.cc:492] T 6dcfa1d900634f87afde19e4dfa48a72 P d869af37b51740718e3ea96f0972bc81: No bootstrap required, opened a new log
I20260619 10:17:14.906703 3141 ts_tablet_manager.cc:1397] T 6dcfa1d900634f87afde19e4dfa48a72 P d869af37b51740718e3ea96f0972bc81: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260619 10:17:14.906695 3137 raft_consensus.cc:348] T 6dcfa1d900634f87afde19e4dfa48a72 P 2b2d8effb39f4473b8bb107821ef1c81 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3a2757b35924642b12335f33832df9d" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 35095 } } peers { permanent_uuid: "d869af37b51740718e3ea96f0972bc81" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 41427 } } peers { permanent_uuid: "2b2d8effb39f4473b8bb107821ef1c81" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 35437 } }
I20260619 10:17:14.906778 3137 raft_consensus.cc:374] T 6dcfa1d900634f87afde19e4dfa48a72 P 2b2d8effb39f4473b8bb107821ef1c81 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 10:17:14.906801 3137 raft_consensus.cc:729] T 6dcfa1d900634f87afde19e4dfa48a72 P 2b2d8effb39f4473b8bb107821ef1c81 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2b2d8effb39f4473b8bb107821ef1c81, State: Initialized, Role: FOLLOWER
I20260619 10:17:14.906839 3137 consensus_queue.cc:260] T 6dcfa1d900634f87afde19e4dfa48a72 P 2b2d8effb39f4473b8bb107821ef1c81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3a2757b35924642b12335f33832df9d" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 35095 } } peers { permanent_uuid: "d869af37b51740718e3ea96f0972bc81" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 41427 } } peers { permanent_uuid: "2b2d8effb39f4473b8bb107821ef1c81" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 35437 } }
I20260619 10:17:14.906841 3141 raft_consensus.cc:348] T 6dcfa1d900634f87afde19e4dfa48a72 P d869af37b51740718e3ea96f0972bc81 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3a2757b35924642b12335f33832df9d" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 35095 } } peers { permanent_uuid: "d869af37b51740718e3ea96f0972bc81" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 41427 } } peers { permanent_uuid: "2b2d8effb39f4473b8bb107821ef1c81" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 35437 } }
I20260619 10:17:14.906908 3141 raft_consensus.cc:374] T 6dcfa1d900634f87afde19e4dfa48a72 P d869af37b51740718e3ea96f0972bc81 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 10:17:14.906921 3137 ts_tablet_manager.cc:1428] T 6dcfa1d900634f87afde19e4dfa48a72 P 2b2d8effb39f4473b8bb107821ef1c81: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260619 10:17:14.906930 3141 raft_consensus.cc:729] T 6dcfa1d900634f87afde19e4dfa48a72 P d869af37b51740718e3ea96f0972bc81 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d869af37b51740718e3ea96f0972bc81, State: Initialized, Role: FOLLOWER
I20260619 10:17:14.906980 3141 consensus_queue.cc:260] T 6dcfa1d900634f87afde19e4dfa48a72 P d869af37b51740718e3ea96f0972bc81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3a2757b35924642b12335f33832df9d" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 35095 } } peers { permanent_uuid: "d869af37b51740718e3ea96f0972bc81" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 41427 } } peers { permanent_uuid: "2b2d8effb39f4473b8bb107821ef1c81" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 35437 } }
I20260619 10:17:14.907061 3141 ts_tablet_manager.cc:1428] T 6dcfa1d900634f87afde19e4dfa48a72 P d869af37b51740718e3ea96f0972bc81: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260619 10:17:15.248867 3056 raft_consensus.cc:482] T 6dcfa1d900634f87afde19e4dfa48a72 P 2b2d8effb39f4473b8bb107821ef1c81 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 10:17:15.248956 3056 raft_consensus.cc:504] T 6dcfa1d900634f87afde19e4dfa48a72 P 2b2d8effb39f4473b8bb107821ef1c81 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3a2757b35924642b12335f33832df9d" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 35095 } } peers { permanent_uuid: "d869af37b51740718e3ea96f0972bc81" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 41427 } } peers { permanent_uuid: "2b2d8effb39f4473b8bb107821ef1c81" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 35437 } }
I20260619 10:17:15.249174 3056 leader_election.cc:288] T 6dcfa1d900634f87afde19e4dfa48a72 P 2b2d8effb39f4473b8bb107821ef1c81 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b3a2757b35924642b12335f33832df9d (127.0.206.65:35095), d869af37b51740718e3ea96f0972bc81 (127.0.206.66:41427)
I20260619 10:17:15.249404 2704 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6dcfa1d900634f87afde19e4dfa48a72" candidate_uuid: "2b2d8effb39f4473b8bb107821ef1c81" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b3a2757b35924642b12335f33832df9d" is_pre_election: true
I20260619 10:17:15.249447 2844 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6dcfa1d900634f87afde19e4dfa48a72" candidate_uuid: "2b2d8effb39f4473b8bb107821ef1c81" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d869af37b51740718e3ea96f0972bc81" is_pre_election: true
I20260619 10:17:15.249511 2704 raft_consensus.cc:2454] T 6dcfa1d900634f87afde19e4dfa48a72 P b3a2757b35924642b12335f33832df9d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2b2d8effb39f4473b8bb107821ef1c81 in term 0.
I20260619 10:17:15.249562 2844 raft_consensus.cc:2454] T 6dcfa1d900634f87afde19e4dfa48a72 P d869af37b51740718e3ea96f0972bc81 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2b2d8effb39f4473b8bb107821ef1c81 in term 0.
I20260619 10:17:15.249768 2916 leader_election.cc:302] T 6dcfa1d900634f87afde19e4dfa48a72 P 2b2d8effb39f4473b8bb107821ef1c81 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2b2d8effb39f4473b8bb107821ef1c81, d869af37b51740718e3ea96f0972bc81; no voters:
I20260619 10:17:15.249869 3056 raft_consensus.cc:2781] T 6dcfa1d900634f87afde19e4dfa48a72 P 2b2d8effb39f4473b8bb107821ef1c81 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 10:17:15.249934 3056 raft_consensus.cc:482] T 6dcfa1d900634f87afde19e4dfa48a72 P 2b2d8effb39f4473b8bb107821ef1c81 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 10:17:15.249967 3056 raft_consensus.cc:3037] T 6dcfa1d900634f87afde19e4dfa48a72 P 2b2d8effb39f4473b8bb107821ef1c81 [term 0 FOLLOWER]: Advancing to term 1
I20260619 10:17:15.250761 3056 raft_consensus.cc:504] T 6dcfa1d900634f87afde19e4dfa48a72 P 2b2d8effb39f4473b8bb107821ef1c81 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3a2757b35924642b12335f33832df9d" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 35095 } } peers { permanent_uuid: "d869af37b51740718e3ea96f0972bc81" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 41427 } } peers { permanent_uuid: "2b2d8effb39f4473b8bb107821ef1c81" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 35437 } }
I20260619 10:17:15.250880 3056 leader_election.cc:288] T 6dcfa1d900634f87afde19e4dfa48a72 P 2b2d8effb39f4473b8bb107821ef1c81 [CANDIDATE]: Term 1 election: Requested vote from peers b3a2757b35924642b12335f33832df9d (127.0.206.65:35095), d869af37b51740718e3ea96f0972bc81 (127.0.206.66:41427)
I20260619 10:17:15.251029 2704 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6dcfa1d900634f87afde19e4dfa48a72" candidate_uuid: "2b2d8effb39f4473b8bb107821ef1c81" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b3a2757b35924642b12335f33832df9d"
I20260619 10:17:15.251046 2844 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6dcfa1d900634f87afde19e4dfa48a72" candidate_uuid: "2b2d8effb39f4473b8bb107821ef1c81" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d869af37b51740718e3ea96f0972bc81"
I20260619 10:17:15.251135 2844 raft_consensus.cc:3037] T 6dcfa1d900634f87afde19e4dfa48a72 P d869af37b51740718e3ea96f0972bc81 [term 0 FOLLOWER]: Advancing to term 1
I20260619 10:17:15.251132 2704 raft_consensus.cc:3037] T 6dcfa1d900634f87afde19e4dfa48a72 P b3a2757b35924642b12335f33832df9d [term 0 FOLLOWER]: Advancing to term 1
I20260619 10:17:15.251768 2704 raft_consensus.cc:2454] T 6dcfa1d900634f87afde19e4dfa48a72 P b3a2757b35924642b12335f33832df9d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2b2d8effb39f4473b8bb107821ef1c81 in term 1.
I20260619 10:17:15.251835 2844 raft_consensus.cc:2454] T 6dcfa1d900634f87afde19e4dfa48a72 P d869af37b51740718e3ea96f0972bc81 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2b2d8effb39f4473b8bb107821ef1c81 in term 1.
I20260619 10:17:15.251960 2916 leader_election.cc:302] T 6dcfa1d900634f87afde19e4dfa48a72 P 2b2d8effb39f4473b8bb107821ef1c81 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2b2d8effb39f4473b8bb107821ef1c81, b3a2757b35924642b12335f33832df9d; no voters:
I20260619 10:17:15.252072 3056 raft_consensus.cc:2781] T 6dcfa1d900634f87afde19e4dfa48a72 P 2b2d8effb39f4473b8bb107821ef1c81 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 10:17:15.252148 3056 raft_consensus.cc:686] T 6dcfa1d900634f87afde19e4dfa48a72 P 2b2d8effb39f4473b8bb107821ef1c81 [term 1 LEADER]: Becoming Leader. State: Replica: 2b2d8effb39f4473b8bb107821ef1c81, State: Running, Role: LEADER
I20260619 10:17:15.252228 3056 consensus_queue.cc:237] T 6dcfa1d900634f87afde19e4dfa48a72 P 2b2d8effb39f4473b8bb107821ef1c81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3a2757b35924642b12335f33832df9d" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 35095 } } peers { permanent_uuid: "d869af37b51740718e3ea96f0972bc81" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 41427 } } peers { permanent_uuid: "2b2d8effb39f4473b8bb107821ef1c81" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 35437 } }
I20260619 10:17:15.252830 2546 catalog_manager.cc:5495] T 6dcfa1d900634f87afde19e4dfa48a72 P 2b2d8effb39f4473b8bb107821ef1c81 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2b2d8effb39f4473b8bb107821ef1c81 (127.0.206.67). New cstate: current_term: 1 leader_uuid: "2b2d8effb39f4473b8bb107821ef1c81" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3a2757b35924642b12335f33832df9d" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 35095 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d869af37b51740718e3ea96f0972bc81" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 41427 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2b2d8effb39f4473b8bb107821ef1c81" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 35437 } health_report { overall_health: HEALTHY } } }
I20260619 10:17:15.261869 2844 raft_consensus.cc:1261] T 6dcfa1d900634f87afde19e4dfa48a72 P d869af37b51740718e3ea96f0972bc81 [term 1 FOLLOWER]: Refusing update from remote peer 2b2d8effb39f4473b8bb107821ef1c81: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 10:17:15.261873 2704 raft_consensus.cc:1261] T 6dcfa1d900634f87afde19e4dfa48a72 P b3a2757b35924642b12335f33832df9d [term 1 FOLLOWER]: Refusing update from remote peer 2b2d8effb39f4473b8bb107821ef1c81: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 10:17:15.262065 3056 consensus_queue.cc:1035] T 6dcfa1d900634f87afde19e4dfa48a72 P 2b2d8effb39f4473b8bb107821ef1c81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d869af37b51740718e3ea96f0972bc81" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 41427 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 10:17:15.262137 3114 consensus_queue.cc:1035] T 6dcfa1d900634f87afde19e4dfa48a72 P 2b2d8effb39f4473b8bb107821ef1c81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b3a2757b35924642b12335f33832df9d" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 35095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260619 10:17:15.266010 3107 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260619 10:17:16.781808 2546 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:36498:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 10:17:16.781945 2546 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:36498:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 10:17:16.782861 2546 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 28ae00894a9046c291da0695c210585d: Sending DeleteTablet for 3 replicas of tablet 6dcfa1d900634f87afde19e4dfa48a72
I20260619 10:17:16.783136 2964 tablet_service.cc:1514] Processing DeleteTablet for tablet 6dcfa1d900634f87afde19e4dfa48a72 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 10:17:16 UTC) from {username='kudu', principal='kudu/127.0.206.126@KRBTEST.COM'} at 127.0.0.1:48628
I20260619 10:17:16.783262 825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTU9OP6/build/release/bin/kudu with pid 2620
I20260619 10:17:16.783325 2825 tablet_service.cc:1514] Processing DeleteTablet for tablet 6dcfa1d900634f87afde19e4dfa48a72 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 10:17:16 UTC) from {username='kudu', principal='kudu/127.0.206.126@KRBTEST.COM'} at 127.0.0.1:51598
I20260619 10:17:16.783756 3176 tablet_replica.cc:330] T 6dcfa1d900634f87afde19e4dfa48a72 P 2b2d8effb39f4473b8bb107821ef1c81: stopping tablet replica
I20260619 10:17:16.783815 3176 raft_consensus.cc:2229] T 6dcfa1d900634f87afde19e4dfa48a72 P 2b2d8effb39f4473b8bb107821ef1c81 [term 1 LEADER]: Raft consensus shutting down.
I20260619 10:17:16.783869 3176 raft_consensus.cc:2258] T 6dcfa1d900634f87afde19e4dfa48a72 P 2b2d8effb39f4473b8bb107821ef1c81 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 10:17:16.784009 3177 tablet_replica.cc:330] T 6dcfa1d900634f87afde19e4dfa48a72 P d869af37b51740718e3ea96f0972bc81: stopping tablet replica
I20260619 10:17:16.784071 3177 raft_consensus.cc:2229] T 6dcfa1d900634f87afde19e4dfa48a72 P d869af37b51740718e3ea96f0972bc81 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 10:17:16.784132 3177 raft_consensus.cc:2258] T 6dcfa1d900634f87afde19e4dfa48a72 P d869af37b51740718e3ea96f0972bc81 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 10:17:16.784127 3176 ts_tablet_manager.cc:1905] T 6dcfa1d900634f87afde19e4dfa48a72 P 2b2d8effb39f4473b8bb107821ef1c81: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 10:17:16.784355 3177 ts_tablet_manager.cc:1905] T 6dcfa1d900634f87afde19e4dfa48a72 P d869af37b51740718e3ea96f0972bc81: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 10:17:16.785789 3177 ts_tablet_manager.cc:1918] T 6dcfa1d900634f87afde19e4dfa48a72 P d869af37b51740718e3ea96f0972bc81: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260619 10:17:16.785869 3177 log.cc:1196] T 6dcfa1d900634f87afde19e4dfa48a72 P d869af37b51740718e3ea96f0972bc81: Deleting WAL directory at /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781864224577058-825-0/minicluster-data/ts-1/wal/wals/6dcfa1d900634f87afde19e4dfa48a72
I20260619 10:17:16.786077 3176 ts_tablet_manager.cc:1918] T 6dcfa1d900634f87afde19e4dfa48a72 P 2b2d8effb39f4473b8bb107821ef1c81: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260619 10:17:16.786104 3177 ts_tablet_manager.cc:1939] T 6dcfa1d900634f87afde19e4dfa48a72 P d869af37b51740718e3ea96f0972bc81: Deleting consensus metadata
I20260619 10:17:16.786177 3176 log.cc:1196] T 6dcfa1d900634f87afde19e4dfa48a72 P 2b2d8effb39f4473b8bb107821ef1c81: Deleting WAL directory at /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781864224577058-825-0/minicluster-data/ts-2/wal/wals/6dcfa1d900634f87afde19e4dfa48a72
I20260619 10:17:16.786410 2532 catalog_manager.cc:4841] TS d869af37b51740718e3ea96f0972bc81 (127.0.206.66:41427): tablet 6dcfa1d900634f87afde19e4dfa48a72 (table test-table [id=9a15f0f8f38842fa9cf2e274fe2d5c38]) successfully deleted
I20260619 10:17:16.786446 3176 ts_tablet_manager.cc:1939] T 6dcfa1d900634f87afde19e4dfa48a72 P 2b2d8effb39f4473b8bb107821ef1c81: Deleting consensus metadata
I20260619 10:17:16.786754 2534 catalog_manager.cc:4841] TS 2b2d8effb39f4473b8bb107821ef1c81 (127.0.206.67:35437): tablet 6dcfa1d900634f87afde19e4dfa48a72 (table test-table [id=9a15f0f8f38842fa9cf2e274fe2d5c38]) successfully deleted
W20260619 10:17:16.790683 2532 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.0.206.65:35095 (error 108)
I20260619 10:17:16.790920 825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTU9OP6/build/release/bin/kudu with pid 2760
I20260619 10:17:16.794783 3179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:16.791042 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.65:35095 (local address 127.0.0.1:57342)
0619 10:17:16.794708 (+ 3666us) negotiation.cc:107] Waiting for socket to connect
0619 10:17:16.794750 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.65:35095: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":3638,"thread_start_us":50,"threads_started":1}
W20260619 10:17:16.794914 2532 catalog_manager.cc:4584] TS b3a2757b35924642b12335f33832df9d (127.0.206.65:35095): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 6dcfa1d900634f87afde19e4dfa48a72: Network error: Client connection negotiation failed: client connection to 127.0.206.65:35095: connect: Connection refused (error 111)
I20260619 10:17:16.796204 825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTU9OP6/build/release/bin/kudu with pid 2900
I20260619 10:17:16.801833 825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTU9OP6/build/release/bin/kudu with pid 2516
2026-06-19T10:17:16Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (5832 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781864224577058-825-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 10:17:16 dist-test-slave-6pt2 krb5kdc[3182](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 10:17:16 dist-test-slave-6pt2 krb5kdc[3182](info): set up 2 sockets
Jun 19 10:17:16 dist-test-slave-6pt2 krb5kdc[3182](info): commencing operation
krb5kdc: starting...
W20260619 10:17:17.850378 825 mini_kdc.cc:121] Time spent starting KDC: real 1.028s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781864224577058-825-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781864224577058-825-0/krb5kdc/test-user.keytab.
Jun 19 10:17:17 dist-test-slave-6pt2 krb5kdc[3182](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864237, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T10:17:17Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T10:17:17Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.206.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.206.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.206.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781864224577058-825-0/krb5kdc/kudu_127.0.206.126.keytab.
Entry for principal kudu/127.0.206.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781864224577058-825-0/krb5kdc/kudu_127.0.206.126.keytab.
I20260619 10:17:17.959640 825 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTU9OP6/build/release/bin/kudu
/tmp/dist-test-taskTU9OP6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781864224577058-825-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781864224577058-825-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781864224577058-825-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781864224577058-825-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.206.126:36845
--webserver_interface=127.0.206.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781864224577058-825-0/krb5kdc/kudu_127.0.206.126.keytab
--principal=kudu/127.0.206.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.206.84:45431
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.206.126:36845
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781864224577058-825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781864224577058-825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781864224577058-825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 10:17:18.036636 3195 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 10:17:18.036859 3195 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 10:17:18.036891 3195 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 10:17:18.038043 3195 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 10:17:18.038107 3195 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 10:17:18.038133 3195 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 10:17:18.038164 3195 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 10:17:18.038187 3195 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 10:17:18.039716 3195 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.206.84:45431
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781864224577058-825-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781864224577058-825-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.206.126:36845
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781864224577058-825-0/krb5kdc/kudu_127.0.206.126.keytab
--principal=kudu/127.0.206.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.206.126:36845
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781864224577058-825-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.206.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3195
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781864224577058-825-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 09:32:44 UTC on ccycloud.quasar-gbtldp.root.comops.site
build id 4455440
I20260619 10:17:18.039983 3195 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 10:17:18.040196 3195 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 10:17:18.044787 3200 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 10:17:18.044857 3201 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 10:17:18.044930 3195 server_base.cc:1034] running on GCE node
W20260619 10:17:18.044836 3203 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 10:17:18.045287 3195 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 10:17:18.045593 3195 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 10:17:18.046747 3195 hybrid_clock.cc:648] HybridClock initialized: now 1781864238046725 us; error 38 us; skew 500 ppm
Jun 19 10:17:18 dist-test-slave-6pt2 krb5kdc[3182](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864238, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 10:17:18.049180 3195 init.cc:377] Logged in from keytab as kudu/127.0.206.126@KRBTEST.COM (short username kudu)
I20260619 10:17:18.049546 3195 webserver.cc:460] Webserver started at http://127.0.206.126:37089/ using document root <none> and password file <none>
I20260619 10:17:18.049746 3195 fs_manager.cc:359] Metadata directory not provided
I20260619 10:17:18.049789 3195 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 10:17:18.049877 3195 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 10:17:18.051017 3195 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781864224577058-825-0/minicluster-data/master-0/data/instance:
uuid: "1cb7acbd85c64b3ba892f8e89c3c1aa2"
format_stamp: "Formatted at 2026-06-19 10:17:18 on dist-test-slave-6pt2"
server_key: "2eb8d9c62d274e647c9bf634d1d01b19"
server_key_iv: "82124328b70b56588227afafbf24890e"
server_key_version: "encryptionkey@0"
I20260619 10:17:18.051313 3195 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781864224577058-825-0/minicluster-data/master-0/wal/instance:
uuid: "1cb7acbd85c64b3ba892f8e89c3c1aa2"
format_stamp: "Formatted at 2026-06-19 10:17:18 on dist-test-slave-6pt2"
server_key: "2eb8d9c62d274e647c9bf634d1d01b19"
server_key_iv: "82124328b70b56588227afafbf24890e"
server_key_version: "encryptionkey@0"
I20260619 10:17:18.052600 3195 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 10:17:18.053354 3210 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 10:17:18.053530 3195 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 10:17:18.053629 3195 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781864224577058-825-0/minicluster-data/master-0/data,/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781864224577058-825-0/minicluster-data/master-0/wal
uuid: "1cb7acbd85c64b3ba892f8e89c3c1aa2"
format_stamp: "Formatted at 2026-06-19 10:17:18 on dist-test-slave-6pt2"
server_key: "2eb8d9c62d274e647c9bf634d1d01b19"
server_key_iv: "82124328b70b56588227afafbf24890e"
server_key_version: "encryptionkey@0"
I20260619 10:17:18.053700 3195 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781864224577058-825-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781864224577058-825-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781864224577058-825-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 10:17:18.094333 3195 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 10:17:18.096230 3195 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 10:17:18.096382 3195 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 10:17:18.100533 3195 rpc_server.cc:307] RPC server started. Bound to: 127.0.206.126:36845
I20260619 10:17:18.100602 3262 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.206.126:36845 every 8 connection(s)
I20260619 10:17:18.101146 3195 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781864224577058-825-0/minicluster-data/master-0/data/info.pb
I20260619 10:17:18.101807 3263 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 10:17:18.103714 825 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTU9OP6/build/release/bin/kudu as pid 3195
I20260619 10:17:18.103813 825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781864224577058-825-0/minicluster-data/master-0/wal/instance
I20260619 10:17:18.103964 825 external_mini_cluster.cc:1428] Setting key 0492f3ec070d644e56b1dc1efbfa3133
I20260619 10:17:18.104282 3263 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1cb7acbd85c64b3ba892f8e89c3c1aa2: Bootstrap starting.
I20260619 10:17:18.104948 3263 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1cb7acbd85c64b3ba892f8e89c3c1aa2: Neither blocks nor log segments found. Creating new log.
I20260619 10:17:18.105240 3263 log.cc:824] T 00000000000000000000000000000000 P 1cb7acbd85c64b3ba892f8e89c3c1aa2: Log is configured to *not* fsync() on all Append() calls
I20260619 10:17:18.106001 3263 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1cb7acbd85c64b3ba892f8e89c3c1aa2: No bootstrap required, opened a new log
I20260619 10:17:18.108191 3263 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1cb7acbd85c64b3ba892f8e89c3c1aa2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1cb7acbd85c64b3ba892f8e89c3c1aa2" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 36845 } }
I20260619 10:17:18.108361 3263 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1cb7acbd85c64b3ba892f8e89c3c1aa2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 10:17:18.108403 3263 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1cb7acbd85c64b3ba892f8e89c3c1aa2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1cb7acbd85c64b3ba892f8e89c3c1aa2, State: Initialized, Role: FOLLOWER
I20260619 10:17:18.108546 3263 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1cb7acbd85c64b3ba892f8e89c3c1aa2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1cb7acbd85c64b3ba892f8e89c3c1aa2" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 36845 } }
I20260619 10:17:18.108610 3263 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1cb7acbd85c64b3ba892f8e89c3c1aa2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 10:17:18.108631 3263 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1cb7acbd85c64b3ba892f8e89c3c1aa2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 10:17:18.108706 3263 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1cb7acbd85c64b3ba892f8e89c3c1aa2 [term 0 FOLLOWER]: Advancing to term 1
I20260619 10:17:18.109256 3263 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1cb7acbd85c64b3ba892f8e89c3c1aa2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1cb7acbd85c64b3ba892f8e89c3c1aa2" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 36845 } }
I20260619 10:17:18.109364 3263 leader_election.cc:302] T 00000000000000000000000000000000 P 1cb7acbd85c64b3ba892f8e89c3c1aa2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1cb7acbd85c64b3ba892f8e89c3c1aa2; no voters:
I20260619 10:17:18.109543 3263 leader_election.cc:288] T 00000000000000000000000000000000 P 1cb7acbd85c64b3ba892f8e89c3c1aa2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 10:17:18.109640 3268 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1cb7acbd85c64b3ba892f8e89c3c1aa2 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 10:17:18.109790 3268 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1cb7acbd85c64b3ba892f8e89c3c1aa2 [term 1 LEADER]: Becoming Leader. State: Replica: 1cb7acbd85c64b3ba892f8e89c3c1aa2, State: Running, Role: LEADER
I20260619 10:17:18.109820 3263 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1cb7acbd85c64b3ba892f8e89c3c1aa2 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 10:17:18.109992 3268 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1cb7acbd85c64b3ba892f8e89c3c1aa2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1cb7acbd85c64b3ba892f8e89c3c1aa2" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 36845 } }
I20260619 10:17:18.110365 3269 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1cb7acbd85c64b3ba892f8e89c3c1aa2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1cb7acbd85c64b3ba892f8e89c3c1aa2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1cb7acbd85c64b3ba892f8e89c3c1aa2" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 36845 } } }
I20260619 10:17:18.110359 3268 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1cb7acbd85c64b3ba892f8e89c3c1aa2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1cb7acbd85c64b3ba892f8e89c3c1aa2. Latest consensus state: current_term: 1 leader_uuid: "1cb7acbd85c64b3ba892f8e89c3c1aa2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1cb7acbd85c64b3ba892f8e89c3c1aa2" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 36845 } } }
I20260619 10:17:18.110491 3269 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1cb7acbd85c64b3ba892f8e89c3c1aa2 [sys.catalog]: This master's current role is: LEADER
I20260619 10:17:18.110563 3268 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1cb7acbd85c64b3ba892f8e89c3c1aa2 [sys.catalog]: This master's current role is: LEADER
I20260619 10:17:18.110805 3277 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 10:17:18.111270 3277 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Jun 19 10:17:18 dist-test-slave-6pt2 krb5kdc[3182](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864237, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.206.126@KRBTEST.COM
I20260619 10:17:18.113220 3277 catalog_manager.cc:1347] Generated new cluster ID: 7c8dcc3f605e4154b16a2d20e2886566
I20260619 10:17:18.113317 3277 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 10:17:18.115924 3266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:18.104467 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42536 (local address 127.0.206.126:36845)
0619 10:17:18.104642 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0619 10:17:18.104649 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 10:17:18.104674 (+ 25us) server_negotiation.cc:407] Connection header received
0619 10:17:18.105149 (+ 475us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:17:18.105156 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:17:18.105202 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:17:18.105336 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:17:18.106114 (+ 778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:18.107209 (+ 1095us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:18.108398 (+ 1189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:18.109136 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:18.109222 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:18.112284 (+ 3062us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 10:17:18.112287 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 10:17:18.112288 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 10:17:18.112297 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0619 10:17:18.114286 (+ 1989us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:17:18.114690 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:17:18.114694 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:17:18.114694 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:17:18.114800 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:17:18.115004 (+ 204us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:17:18.115005 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:17:18.115005 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:17:18.115310 (+ 305us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 10:17:18.115455 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0619 10:17:18.115570 (+ 115us) server_negotiation.cc:299] Negotiation successful
0619 10:17:18.115708 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":125,"thread_start_us":83,"threads_started":1}
I20260619 10:17:18.135773 3277 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 10:17:18.136317 3277 catalog_manager.cc:1504] Loading token signing keys...
I20260619 10:17:18.138058 3277 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1cb7acbd85c64b3ba892f8e89c3c1aa2: Generated new TSK 0
I20260619 10:17:18.138243 3277 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.0.206.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.206.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.206.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781864224577058-825-0/krb5kdc/kudu_127.0.206.65.keytab.
Entry for principal kudu/127.0.206.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781864224577058-825-0/krb5kdc/kudu_127.0.206.65.keytab.
I20260619 10:17:18.166553 825 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTU9OP6/build/release/bin/kudu
/tmp/dist-test-taskTU9OP6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781864224577058-825-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781864224577058-825-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781864224577058-825-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781864224577058-825-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.206.65:0
--local_ip_for_outbound_sockets=127.0.206.65
--webserver_interface=127.0.206.65
--webserver_port=0
--tserver_master_addrs=127.0.206.126:36845
--keytab_file=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781864224577058-825-0/krb5kdc/kudu_127.0.206.65.keytab
--principal=kudu/127.0.206.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.206.84:45431
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781864224577058-825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781864224577058-825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781864224577058-825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 10:17:18.248955 3289 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 10:17:18.249166 3289 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 10:17:18.249204 3289 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 10:17:18.250350 3289 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 10:17:18.250413 3289 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 10:17:18.250458 3289 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.206.65
I20260619 10:17:18.252007 3289 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.206.84:45431
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781864224577058-825-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781864224577058-825-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781864224577058-825-0/krb5kdc/kudu_127.0.206.65.keytab
--principal=kudu/127.0.206.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.206.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781864224577058-825-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.206.65
--webserver_port=0
--tserver_master_addrs=127.0.206.126:36845
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3289
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.206.65
--log_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781864224577058-825-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 09:32:44 UTC on ccycloud.quasar-gbtldp.root.comops.site
build id 4455440
I20260619 10:17:18.252278 3289 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 10:17:18.252496 3289 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 10:17:18.257121 3297 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 10:17:18.257222 3295 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 10:17:18.257272 3294 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 10:17:18.257318 3289 server_base.cc:1034] running on GCE node
I20260619 10:17:18.257691 3289 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 10:17:18.257928 3289 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 10:17:18.259059 3289 hybrid_clock.cc:648] HybridClock initialized: now 1781864238259045 us; error 32 us; skew 500 ppm
Jun 19 10:17:18 dist-test-slave-6pt2 krb5kdc[3182](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864238, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 10:17:18.261567 3289 init.cc:377] Logged in from keytab as kudu/127.0.206.65@KRBTEST.COM (short username kudu)
I20260619 10:17:18.262022 3289 webserver.cc:460] Webserver started at http://127.0.206.65:42637/ using document root <none> and password file <none>
I20260619 10:17:18.262230 3289 fs_manager.cc:359] Metadata directory not provided
I20260619 10:17:18.262274 3289 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 10:17:18.262441 3289 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 10:17:18.263463 3289 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781864224577058-825-0/minicluster-data/ts-0/data/instance:
uuid: "4b1d1b24a3054e6cb5fe1b8ec25de8c5"
format_stamp: "Formatted at 2026-06-19 10:17:18 on dist-test-slave-6pt2"
server_key: "d282bcf1138655c3b5259acc89f96090"
server_key_iv: "363385479bfb6b1d5360c10c1b10fd3f"
server_key_version: "encryptionkey@0"
I20260619 10:17:18.263818 3289 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781864224577058-825-0/minicluster-data/ts-0/wal/instance:
uuid: "4b1d1b24a3054e6cb5fe1b8ec25de8c5"
format_stamp: "Formatted at 2026-06-19 10:17:18 on dist-test-slave-6pt2"
server_key: "d282bcf1138655c3b5259acc89f96090"
server_key_iv: "363385479bfb6b1d5360c10c1b10fd3f"
server_key_version: "encryptionkey@0"
I20260619 10:17:18.265053 3289 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 10:17:18.265729 3304 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 10:17:18.265888 3289 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 10:17:18.265985 3289 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781864224577058-825-0/minicluster-data/ts-0/data,/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781864224577058-825-0/minicluster-data/ts-0/wal
uuid: "4b1d1b24a3054e6cb5fe1b8ec25de8c5"
format_stamp: "Formatted at 2026-06-19 10:17:18 on dist-test-slave-6pt2"
server_key: "d282bcf1138655c3b5259acc89f96090"
server_key_iv: "363385479bfb6b1d5360c10c1b10fd3f"
server_key_version: "encryptionkey@0"
I20260619 10:17:18.266065 3289 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781864224577058-825-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781864224577058-825-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781864224577058-825-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 10:17:18.275944 3289 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 10:17:18.278232 3289 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 10:17:18.278389 3289 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 10:17:18.278615 3289 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 10:17:18.278913 3289 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 10:17:18.278975 3289 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 10:17:18.279014 3289 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 10:17:18.279042 3289 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 10:17:18.284916 3289 rpc_server.cc:307] RPC server started. Bound to: 127.0.206.65:39931
I20260619 10:17:18.284960 3417 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.206.65:39931 every 8 connection(s)
I20260619 10:17:18.285333 3289 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781864224577058-825-0/minicluster-data/ts-0/data/info.pb
Jun 19 10:17:18 dist-test-slave-6pt2 krb5kdc[3182](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864238, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.65@KRBTEST.COM for kudu/127.0.206.126@KRBTEST.COM
I20260619 10:17:18.291167 825 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTU9OP6/build/release/bin/kudu as pid 3289
I20260619 10:17:18.291231 825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781864224577058-825-0/minicluster-data/ts-0/wal/instance
I20260619 10:17:18.291404 825 external_mini_cluster.cc:1428] Setting key f8a896db39ac7fe99f0fb0e6a3d34aba
I20260619 10:17:18.294250 3266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:18.286002 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.65:34263 (local address 127.0.206.126:36845)
0619 10:17:18.286042 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0619 10:17:18.286044 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 10:17:18.286216 (+ 172us) server_negotiation.cc:407] Connection header received
0619 10:17:18.287196 (+ 980us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:17:18.287197 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:17:18.287206 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:17:18.287254 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:17:18.287770 (+ 516us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:18.288266 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:18.289197 (+ 931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:18.289383 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:18.289429 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:18.291366 (+ 1937us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 10:17:18.291367 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 10:17:18.291368 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 10:17:18.291375 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 10:17:18.292858 (+ 1483us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:17:18.293354 (+ 496us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:17:18.293355 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:17:18.293355 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:17:18.293406 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:17:18.293668 (+ 262us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:17:18.293669 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:17:18.293669 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:17:18.293772 (+ 103us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 10:17:18.293820 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0619 10:17:18.294119 (+ 299us) server_negotiation.cc:299] Negotiation successful
0619 10:17:18.294190 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260619 10:17:18.294497 3420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:18.285977 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:36845 (local address 127.0.206.65:34263)
0619 10:17:18.286116 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0619 10:17:18.286145 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0619 10:17:18.287096 (+ 951us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 10:17:18.287303 (+ 207us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 10:17:18.287306 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 10:17:18.287531 (+ 225us) client_negotiation.cc:189] Negotiated authn=SASL
0619 10:17:18.287704 (+ 173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:18.287707 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:18.288337 (+ 630us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:18.288338 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:18.289112 (+ 774us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:18.289126 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:18.289448 (+ 322us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:18.289449 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:18.289539 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:18.289955 (+ 416us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 10:17:18.289963 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 10:17:18.291249 (+ 1286us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 10:17:18.292958 (+ 1709us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 10:17:18.292965 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 10:17:18.292967 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 10:17:18.293267 (+ 300us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 10:17:18.293472 (+ 205us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 10:17:18.293473 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 10:17:18.293473 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 10:17:18.293606 (+ 133us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 10:17:18.293834 (+ 228us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 10:17:18.293839 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 10:17:18.293990 (+ 151us) client_negotiation.cc:769] Sending connection context
0619 10:17:18.294107 (+ 117us) client_negotiation.cc:240] Negotiation successful
0619 10:17:18.294274 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":87,"threads_started":1}
I20260619 10:17:18.294966 3418 heartbeater.cc:347] Connected to a master server at 127.0.206.126:36845
I20260619 10:17:18.295053 3418 heartbeater.cc:464] Registering TS with master...
I20260619 10:17:18.295229 3418 heartbeater.cc:511] Master 127.0.206.126:36845 requested a full tablet report, sending...
I20260619 10:17:18.295694 3227 ts_manager.cc:194] Registered new tserver with Master: 4b1d1b24a3054e6cb5fe1b8ec25de8c5 (127.0.206.65:39931)
I20260619 10:17:18.296247 3227 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.206.65@KRBTEST.COM'} at 127.0.206.65:34263
WARNING: no policy specified for kudu/127.0.206.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.206.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.206.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781864224577058-825-0/krb5kdc/kudu_127.0.206.66.keytab.
Entry for principal kudu/127.0.206.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781864224577058-825-0/krb5kdc/kudu_127.0.206.66.keytab.
I20260619 10:17:18.314249 825 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTU9OP6/build/release/bin/kudu
/tmp/dist-test-taskTU9OP6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781864224577058-825-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781864224577058-825-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781864224577058-825-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781864224577058-825-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.206.66:0
--local_ip_for_outbound_sockets=127.0.206.66
--webserver_interface=127.0.206.66
--webserver_port=0
--tserver_master_addrs=127.0.206.126:36845
--keytab_file=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781864224577058-825-0/krb5kdc/kudu_127.0.206.66.keytab
--principal=kudu/127.0.206.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.206.84:45431
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781864224577058-825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781864224577058-825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781864224577058-825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 10:17:18.390797 3423 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 10:17:18.390980 3423 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 10:17:18.391000 3423 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 10:17:18.392148 3423 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 10:17:18.392194 3423 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 10:17:18.392220 3423 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.206.66
I20260619 10:17:18.393668 3423 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.206.84:45431
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781864224577058-825-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781864224577058-825-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781864224577058-825-0/krb5kdc/kudu_127.0.206.66.keytab
--principal=kudu/127.0.206.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.206.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781864224577058-825-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.206.66
--webserver_port=0
--tserver_master_addrs=127.0.206.126:36845
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3423
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.206.66
--log_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781864224577058-825-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 09:32:44 UTC on ccycloud.quasar-gbtldp.root.comops.site
build id 4455440
I20260619 10:17:18.393882 3423 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 10:17:18.394093 3423 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 10:17:18.398932 3429 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 10:17:18.398885 3428 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 10:17:18.398896 3431 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 10:17:18.398917 3423 server_base.cc:1034] running on GCE node
I20260619 10:17:18.399302 3423 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 10:17:18.399549 3423 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 10:17:18.400707 3423 hybrid_clock.cc:648] HybridClock initialized: now 1781864238400690 us; error 30 us; skew 500 ppm
Jun 19 10:17:18 dist-test-slave-6pt2 krb5kdc[3182](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864238, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 10:17:18.403148 3423 init.cc:377] Logged in from keytab as kudu/127.0.206.66@KRBTEST.COM (short username kudu)
I20260619 10:17:18.403576 3423 webserver.cc:460] Webserver started at http://127.0.206.66:38817/ using document root <none> and password file <none>
I20260619 10:17:18.403795 3423 fs_manager.cc:359] Metadata directory not provided
I20260619 10:17:18.403859 3423 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 10:17:18.404001 3423 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 10:17:18.404942 3423 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781864224577058-825-0/minicluster-data/ts-1/data/instance:
uuid: "f2ae9ef9999548f8a82fe597a9dfb912"
format_stamp: "Formatted at 2026-06-19 10:17:18 on dist-test-slave-6pt2"
server_key: "9d0dca79b61cdc82e5ab47131b341fdb"
server_key_iv: "758cbbe26802af8621ac98502e49cf0f"
server_key_version: "encryptionkey@0"
I20260619 10:17:18.405285 3423 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781864224577058-825-0/minicluster-data/ts-1/wal/instance:
uuid: "f2ae9ef9999548f8a82fe597a9dfb912"
format_stamp: "Formatted at 2026-06-19 10:17:18 on dist-test-slave-6pt2"
server_key: "9d0dca79b61cdc82e5ab47131b341fdb"
server_key_iv: "758cbbe26802af8621ac98502e49cf0f"
server_key_version: "encryptionkey@0"
I20260619 10:17:18.406535 3423 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260619 10:17:18.407256 3438 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 10:17:18.407419 3423 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 10:17:18.407511 3423 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781864224577058-825-0/minicluster-data/ts-1/data,/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781864224577058-825-0/minicluster-data/ts-1/wal
uuid: "f2ae9ef9999548f8a82fe597a9dfb912"
format_stamp: "Formatted at 2026-06-19 10:17:18 on dist-test-slave-6pt2"
server_key: "9d0dca79b61cdc82e5ab47131b341fdb"
server_key_iv: "758cbbe26802af8621ac98502e49cf0f"
server_key_version: "encryptionkey@0"
I20260619 10:17:18.407588 3423 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781864224577058-825-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781864224577058-825-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781864224577058-825-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 10:17:18.438967 3423 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 10:17:18.441241 3423 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 10:17:18.441409 3423 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 10:17:18.441668 3423 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 10:17:18.441958 3423 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 10:17:18.442009 3423 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 10:17:18.442049 3423 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 10:17:18.442075 3423 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 10:17:18.447589 3423 rpc_server.cc:307] RPC server started. Bound to: 127.0.206.66:45733
I20260619 10:17:18.447641 3551 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.206.66:45733 every 8 connection(s)
I20260619 10:17:18.447961 3423 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781864224577058-825-0/minicluster-data/ts-1/data/info.pb
I20260619 10:17:18.448450 825 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTU9OP6/build/release/bin/kudu as pid 3423
I20260619 10:17:18.448531 825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781864224577058-825-0/minicluster-data/ts-1/wal/instance
I20260619 10:17:18.448751 825 external_mini_cluster.cc:1428] Setting key b727e0539c36f6a8cf816d39311e35f1
Jun 19 10:17:18 dist-test-slave-6pt2 krb5kdc[3182](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864238, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.66@KRBTEST.COM for kudu/127.0.206.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.0.206.67@KRBTEST.COM; defaulting to no policy
I20260619 10:17:18.458564 3266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:18.448683 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.66:58917 (local address 127.0.206.126:36845)
0619 10:17:18.448734 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0619 10:17:18.448736 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 10:17:18.449105 (+ 369us) server_negotiation.cc:407] Connection header received
0619 10:17:18.450335 (+ 1230us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:17:18.450336 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:17:18.450345 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:17:18.450382 (+ 37us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:17:18.451245 (+ 863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:18.452046 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:18.453076 (+ 1030us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:18.453395 (+ 319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:18.453482 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:18.455870 (+ 2388us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 10:17:18.455871 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 10:17:18.455872 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 10:17:18.455878 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 10:17:18.457384 (+ 1506us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:17:18.457730 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:17:18.457730 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:17:18.457731 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:17:18.457779 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:17:18.457979 (+ 200us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:17:18.457980 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:17:18.457980 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:17:18.458114 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 10:17:18.458220 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0619 10:17:18.458449 (+ 229us) server_negotiation.cc:299] Negotiation successful
0619 10:17:18.458505 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260619 10:17:18.458747 3554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:18.448675 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:36845 (local address 127.0.206.66:58917)
0619 10:17:18.448962 (+ 287us) negotiation.cc:107] Waiting for socket to connect
0619 10:17:18.449022 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0619 10:17:18.450238 (+ 1216us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 10:17:18.450458 (+ 220us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 10:17:18.450462 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 10:17:18.450872 (+ 410us) client_negotiation.cc:189] Negotiated authn=SASL
0619 10:17:18.451151 (+ 279us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:18.451162 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:18.452128 (+ 966us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:18.452129 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:18.452974 (+ 845us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:18.452976 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:18.453499 (+ 523us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:18.453499 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:18.453628 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:18.454104 (+ 476us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 10:17:18.454112 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 10:17:18.455785 (+ 1673us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 10:17:18.457460 (+ 1675us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 10:17:18.457465 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 10:17:18.457466 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 10:17:18.457665 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 10:17:18.457836 (+ 171us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 10:17:18.457837 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 10:17:18.457837 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 10:17:18.457926 (+ 89us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 10:17:18.458215 (+ 289us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 10:17:18.458220 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 10:17:18.458338 (+ 118us) client_negotiation.cc:769] Sending connection context
0619 10:17:18.458418 (+ 80us) client_negotiation.cc:240] Negotiation successful
0619 10:17:18.458526 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":250,"thread_start_us":221,"threads_started":1}
I20260619 10:17:18.459136 3552 heartbeater.cc:347] Connected to a master server at 127.0.206.126:36845
I20260619 10:17:18.459223 3552 heartbeater.cc:464] Registering TS with master...
I20260619 10:17:18.459349 3552 heartbeater.cc:511] Master 127.0.206.126:36845 requested a full tablet report, sending...
I20260619 10:17:18.459725 3227 ts_manager.cc:194] Registered new tserver with Master: f2ae9ef9999548f8a82fe597a9dfb912 (127.0.206.66:45733)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.206.67@KRBTEST.COM" created.
I20260619 10:17:18.460245 3227 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.206.66@KRBTEST.COM'} at 127.0.206.66:58917
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.206.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781864224577058-825-0/krb5kdc/kudu_127.0.206.67.keytab.
Entry for principal kudu/127.0.206.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781864224577058-825-0/krb5kdc/kudu_127.0.206.67.keytab.
I20260619 10:17:18.473052 825 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTU9OP6/build/release/bin/kudu
/tmp/dist-test-taskTU9OP6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781864224577058-825-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781864224577058-825-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781864224577058-825-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781864224577058-825-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.206.67:0
--local_ip_for_outbound_sockets=127.0.206.67
--webserver_interface=127.0.206.67
--webserver_port=0
--tserver_master_addrs=127.0.206.126:36845
--keytab_file=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781864224577058-825-0/krb5kdc/kudu_127.0.206.67.keytab
--principal=kudu/127.0.206.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.206.84:45431
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781864224577058-825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781864224577058-825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781864224577058-825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 10:17:18.550459 3557 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 10:17:18.550662 3557 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 10:17:18.550685 3557 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 10:17:18.551723 3557 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 10:17:18.551753 3557 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 10:17:18.551777 3557 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.206.67
I20260619 10:17:18.553162 3557 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.206.84:45431
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781864224577058-825-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781864224577058-825-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781864224577058-825-0/krb5kdc/kudu_127.0.206.67.keytab
--principal=kudu/127.0.206.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.206.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781864224577058-825-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.206.67
--webserver_port=0
--tserver_master_addrs=127.0.206.126:36845
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3557
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.206.67
--log_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781864224577058-825-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 09:32:44 UTC on ccycloud.quasar-gbtldp.root.comops.site
build id 4455440
I20260619 10:17:18.553406 3557 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 10:17:18.553623 3557 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 10:17:18.558482 3563 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 10:17:18.558557 3562 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 10:17:18.558580 3557 server_base.cc:1034] running on GCE node
W20260619 10:17:18.558703 3565 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 10:17:18.559000 3557 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 10:17:18.559248 3557 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 10:17:18.560408 3557 hybrid_clock.cc:648] HybridClock initialized: now 1781864238560395 us; error 27 us; skew 500 ppm
Jun 19 10:17:18 dist-test-slave-6pt2 krb5kdc[3182](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864238, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 10:17:18.562894 3557 init.cc:377] Logged in from keytab as kudu/127.0.206.67@KRBTEST.COM (short username kudu)
I20260619 10:17:18.563352 3557 webserver.cc:460] Webserver started at http://127.0.206.67:44823/ using document root <none> and password file <none>
I20260619 10:17:18.563565 3557 fs_manager.cc:359] Metadata directory not provided
I20260619 10:17:18.563627 3557 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 10:17:18.563755 3557 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 10:17:18.564733 3557 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781864224577058-825-0/minicluster-data/ts-2/data/instance:
uuid: "8b12450d5c1f4c6da644c5918e34c268"
format_stamp: "Formatted at 2026-06-19 10:17:18 on dist-test-slave-6pt2"
server_key: "6f8a520f8aecbbeff618bf5cbe3f2983"
server_key_iv: "276bd702edc1dd930406bae74f837d2d"
server_key_version: "encryptionkey@0"
I20260619 10:17:18.565075 3557 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781864224577058-825-0/minicluster-data/ts-2/wal/instance:
uuid: "8b12450d5c1f4c6da644c5918e34c268"
format_stamp: "Formatted at 2026-06-19 10:17:18 on dist-test-slave-6pt2"
server_key: "6f8a520f8aecbbeff618bf5cbe3f2983"
server_key_iv: "276bd702edc1dd930406bae74f837d2d"
server_key_version: "encryptionkey@0"
I20260619 10:17:18.566282 3557 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260619 10:17:18.567040 3572 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 10:17:18.567209 3557 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260619 10:17:18.567301 3557 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781864224577058-825-0/minicluster-data/ts-2/data,/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781864224577058-825-0/minicluster-data/ts-2/wal
uuid: "8b12450d5c1f4c6da644c5918e34c268"
format_stamp: "Formatted at 2026-06-19 10:17:18 on dist-test-slave-6pt2"
server_key: "6f8a520f8aecbbeff618bf5cbe3f2983"
server_key_iv: "276bd702edc1dd930406bae74f837d2d"
server_key_version: "encryptionkey@0"
I20260619 10:17:18.567386 3557 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781864224577058-825-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781864224577058-825-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781864224577058-825-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 10:17:18.578630 3557 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 10:17:18.581312 3557 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 10:17:18.581480 3557 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 10:17:18.581677 3557 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 10:17:18.581943 3557 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 10:17:18.581992 3557 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 10:17:18.582029 3557 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 10:17:18.582055 3557 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 10:17:18.587638 3557 rpc_server.cc:307] RPC server started. Bound to: 127.0.206.67:38167
I20260619 10:17:18.587702 3685 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.206.67:38167 every 8 connection(s)
I20260619 10:17:18.587985 3557 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781864224577058-825-0/minicluster-data/ts-2/data/info.pb
Jun 19 10:17:18 dist-test-slave-6pt2 krb5kdc[3182](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864238, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.67@KRBTEST.COM for kudu/127.0.206.126@KRBTEST.COM
I20260619 10:17:18.596529 3266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:18.588689 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.67:34955 (local address 127.0.206.126:36845)
0619 10:17:18.588737 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0619 10:17:18.588739 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 10:17:18.588975 (+ 236us) server_negotiation.cc:407] Connection header received
0619 10:17:18.589880 (+ 905us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:17:18.589882 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:17:18.589893 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:17:18.589952 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:17:18.590508 (+ 556us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:18.591076 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:18.592071 (+ 995us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:18.592271 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:18.592355 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:18.594278 (+ 1923us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 10:17:18.594279 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 10:17:18.594280 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 10:17:18.594285 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0619 10:17:18.595427 (+ 1142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:17:18.595786 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:17:18.595787 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:17:18.595787 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:17:18.595831 (+ 44us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:17:18.596016 (+ 185us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:17:18.596016 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:17:18.596017 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:17:18.596124 (+ 107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 10:17:18.596179 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0619 10:17:18.596387 (+ 208us) server_negotiation.cc:299] Negotiation successful
0619 10:17:18.596452 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260619 10:17:18.596725 3688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:18.588667 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:36845 (local address 127.0.206.67:34955)
0619 10:17:18.588832 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0619 10:17:18.588862 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0619 10:17:18.589777 (+ 915us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 10:17:18.590014 (+ 237us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 10:17:18.590023 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 10:17:18.590250 (+ 227us) client_negotiation.cc:189] Negotiated authn=SASL
0619 10:17:18.590416 (+ 166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:18.590419 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:18.591179 (+ 760us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:18.591180 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:18.591927 (+ 747us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:18.591929 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:18.592363 (+ 434us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:18.592364 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:18.592480 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:18.592941 (+ 461us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 10:17:18.592948 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0619 10:17:18.594199 (+ 1251us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 10:17:18.595490 (+ 1291us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 10:17:18.595495 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 10:17:18.595497 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 10:17:18.595718 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 10:17:18.595882 (+ 164us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 10:17:18.595883 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 10:17:18.595883 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 10:17:18.595967 (+ 84us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 10:17:18.596182 (+ 215us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 10:17:18.596185 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 10:17:18.596286 (+ 101us) client_negotiation.cc:769] Sending connection context
0619 10:17:18.596363 (+ 77us) client_negotiation.cc:240] Negotiation successful
0619 10:17:18.596497 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":88,"threads_started":1}
I20260619 10:17:18.597245 3686 heartbeater.cc:347] Connected to a master server at 127.0.206.126:36845
I20260619 10:17:18.597337 3686 heartbeater.cc:464] Registering TS with master...
I20260619 10:17:18.597329 825 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTU9OP6/build/release/bin/kudu as pid 3557
I20260619 10:17:18.597407 825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781864224577058-825-0/minicluster-data/ts-2/wal/instance
I20260619 10:17:18.597512 3686 heartbeater.cc:511] Master 127.0.206.126:36845 requested a full tablet report, sending...
I20260619 10:17:18.597595 825 external_mini_cluster.cc:1428] Setting key 45a07825a0c691c5dc329576941503a9
I20260619 10:17:18.597863 3227 ts_manager.cc:194] Registered new tserver with Master: 8b12450d5c1f4c6da644c5918e34c268 (127.0.206.67:38167)
I20260619 10:17:18.598336 3227 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.206.67@KRBTEST.COM'} at 127.0.206.67:34955
I20260619 10:17:18.598428 825 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260619 10:17:18.610232 3266 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 10:17:18.609118 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42542 (local address 127.0.206.126:36845)
0619 10:17:18.609176 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0619 10:17:18.609179 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 10:17:18.609333 (+ 154us) server_negotiation.cc:407] Connection header received
0619 10:17:18.609516 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:17:18.609517 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:17:18.609529 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:17:18.609588 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:17:18.610175 (+ 587us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42542: BlockingRecv error: recv got EOF from 127.0.0.1:42542 (error 108)
Metrics: {"server-negotiator.queue_time_us":15}
I20260619 10:17:18.611568 825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTU9OP6/build/release/bin/kudu with pid 3289
I20260619 10:17:18.616303 825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTU9OP6/build/release/bin/kudu with pid 3423
I20260619 10:17:18.620373 825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTU9OP6/build/release/bin/kudu with pid 3557
I20260619 10:17:18.624647 825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTU9OP6/build/release/bin/kudu with pid 3195
2026-06-19T10:17:18Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (1821 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781864224577058-825-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 10:17:18 dist-test-slave-6pt2 krb5kdc[3698](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 10:17:18 dist-test-slave-6pt2 krb5kdc[3698](info): set up 2 sockets
Jun 19 10:17:18 dist-test-slave-6pt2 krb5kdc[3698](info): commencing operation
krb5kdc: starting...
W20260619 10:17:19.634447 825 mini_kdc.cc:121] Time spent starting KDC: real 0.990s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781864224577058-825-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781864224577058-825-0/krb5kdc/test-user.keytab.
Jun 19 10:17:19 dist-test-slave-6pt2 krb5kdc[3698](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864239, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T10:17:19Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T10:17:19Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.206.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.206.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.206.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781864224577058-825-0/krb5kdc/kudu_127.0.206.126.keytab.
Entry for principal kudu/127.0.206.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781864224577058-825-0/krb5kdc/kudu_127.0.206.126.keytab.
I20260619 10:17:19.742549 825 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTU9OP6/build/release/bin/kudu
/tmp/dist-test-taskTU9OP6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781864224577058-825-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781864224577058-825-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781864224577058-825-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781864224577058-825-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.206.126:38323
--webserver_interface=127.0.206.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781864224577058-825-0/krb5kdc/kudu_127.0.206.126.keytab
--principal=kudu/127.0.206.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.206.84:46241
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.206.126:38323
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781864224577058-825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781864224577058-825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781864224577058-825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 10:17:19.820845 3711 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 10:17:19.821033 3711 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 10:17:19.821054 3711 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 10:17:19.822117 3711 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 10:17:19.822150 3711 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 10:17:19.822165 3711 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 10:17:19.822176 3711 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 10:17:19.822188 3711 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 10:17:19.823596 3711 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.206.84:46241
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781864224577058-825-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781864224577058-825-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.206.126:38323
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781864224577058-825-0/krb5kdc/kudu_127.0.206.126.keytab
--principal=kudu/127.0.206.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.206.126:38323
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781864224577058-825-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.206.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3711
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781864224577058-825-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 09:32:44 UTC on ccycloud.quasar-gbtldp.root.comops.site
build id 4455440
I20260619 10:17:19.823771 3711 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 10:17:19.823925 3711 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 10:17:19.828583 3719 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 10:17:19.828624 3711 server_base.cc:1034] running on GCE node
W20260619 10:17:19.828559 3716 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 10:17:19.828591 3717 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 10:17:19.828958 3711 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 10:17:19.829236 3711 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 10:17:19.830390 3711 hybrid_clock.cc:648] HybridClock initialized: now 1781864239830368 us; error 34 us; skew 500 ppm
Jun 19 10:17:19 dist-test-slave-6pt2 krb5kdc[3698](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864239, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 10:17:19.832736 3711 init.cc:377] Logged in from keytab as kudu/127.0.206.126@KRBTEST.COM (short username kudu)
I20260619 10:17:19.833093 3711 webserver.cc:460] Webserver started at http://127.0.206.126:39723/ using document root <none> and password file <none>
I20260619 10:17:19.833256 3711 fs_manager.cc:359] Metadata directory not provided
I20260619 10:17:19.833293 3711 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 10:17:19.833379 3711 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 10:17:19.834295 3711 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781864224577058-825-0/minicluster-data/master-0/data/instance:
uuid: "76bf52b0ac5147ffb2dac31e2b6f19ad"
format_stamp: "Formatted at 2026-06-19 10:17:19 on dist-test-slave-6pt2"
server_key: "32104e15c52948076a6becfbe8022f57"
server_key_iv: "a6cf979befa89e19560b97865c3210ed"
server_key_version: "encryptionkey@0"
I20260619 10:17:19.834568 3711 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781864224577058-825-0/minicluster-data/master-0/wal/instance:
uuid: "76bf52b0ac5147ffb2dac31e2b6f19ad"
format_stamp: "Formatted at 2026-06-19 10:17:19 on dist-test-slave-6pt2"
server_key: "32104e15c52948076a6becfbe8022f57"
server_key_iv: "a6cf979befa89e19560b97865c3210ed"
server_key_version: "encryptionkey@0"
I20260619 10:17:19.835824 3711 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260619 10:17:19.836470 3726 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 10:17:19.836627 3711 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260619 10:17:19.836709 3711 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781864224577058-825-0/minicluster-data/master-0/data,/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781864224577058-825-0/minicluster-data/master-0/wal
uuid: "76bf52b0ac5147ffb2dac31e2b6f19ad"
format_stamp: "Formatted at 2026-06-19 10:17:19 on dist-test-slave-6pt2"
server_key: "32104e15c52948076a6becfbe8022f57"
server_key_iv: "a6cf979befa89e19560b97865c3210ed"
server_key_version: "encryptionkey@0"
I20260619 10:17:19.836774 3711 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781864224577058-825-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781864224577058-825-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781864224577058-825-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 10:17:19.845220 3711 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 10:17:19.847369 3711 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 10:17:19.847519 3711 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 10:17:19.851079 3711 rpc_server.cc:307] RPC server started. Bound to: 127.0.206.126:38323
I20260619 10:17:19.851142 3778 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.206.126:38323 every 8 connection(s)
I20260619 10:17:19.851460 3711 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781864224577058-825-0/minicluster-data/master-0/data/info.pb
I20260619 10:17:19.852236 3779 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 10:17:19.854209 3779 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 76bf52b0ac5147ffb2dac31e2b6f19ad: Bootstrap starting.
I20260619 10:17:19.854846 3779 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 76bf52b0ac5147ffb2dac31e2b6f19ad: Neither blocks nor log segments found. Creating new log.
I20260619 10:17:19.855068 3779 log.cc:824] T 00000000000000000000000000000000 P 76bf52b0ac5147ffb2dac31e2b6f19ad: Log is configured to *not* fsync() on all Append() calls
I20260619 10:17:19.855619 3779 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 76bf52b0ac5147ffb2dac31e2b6f19ad: No bootstrap required, opened a new log
I20260619 10:17:19.856777 825 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTU9OP6/build/release/bin/kudu as pid 3711
I20260619 10:17:19.856904 825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781864224577058-825-0/minicluster-data/master-0/wal/instance
I20260619 10:17:19.857080 825 external_mini_cluster.cc:1428] Setting key 183a643fef03622d4041c6d1c228057d
I20260619 10:17:19.857259 3779 raft_consensus.cc:348] T 00000000000000000000000000000000 P 76bf52b0ac5147ffb2dac31e2b6f19ad [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "76bf52b0ac5147ffb2dac31e2b6f19ad" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 38323 } }
I20260619 10:17:19.857424 3779 raft_consensus.cc:374] T 00000000000000000000000000000000 P 76bf52b0ac5147ffb2dac31e2b6f19ad [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 10:17:19.857957 3779 raft_consensus.cc:729] T 00000000000000000000000000000000 P 76bf52b0ac5147ffb2dac31e2b6f19ad [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 76bf52b0ac5147ffb2dac31e2b6f19ad, State: Initialized, Role: FOLLOWER
I20260619 10:17:19.858111 3779 consensus_queue.cc:260] T 00000000000000000000000000000000 P 76bf52b0ac5147ffb2dac31e2b6f19ad [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "76bf52b0ac5147ffb2dac31e2b6f19ad" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 38323 } }
I20260619 10:17:19.858198 3779 raft_consensus.cc:388] T 00000000000000000000000000000000 P 76bf52b0ac5147ffb2dac31e2b6f19ad [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 10:17:19.858240 3779 raft_consensus.cc:482] T 00000000000000000000000000000000 P 76bf52b0ac5147ffb2dac31e2b6f19ad [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 10:17:19.858286 3779 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 76bf52b0ac5147ffb2dac31e2b6f19ad [term 0 FOLLOWER]: Advancing to term 1
I20260619 10:17:19.858974 3779 raft_consensus.cc:504] T 00000000000000000000000000000000 P 76bf52b0ac5147ffb2dac31e2b6f19ad [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "76bf52b0ac5147ffb2dac31e2b6f19ad" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 38323 } }
I20260619 10:17:19.859088 3779 leader_election.cc:302] T 00000000000000000000000000000000 P 76bf52b0ac5147ffb2dac31e2b6f19ad [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 76bf52b0ac5147ffb2dac31e2b6f19ad; no voters:
I20260619 10:17:19.859239 3779 leader_election.cc:288] T 00000000000000000000000000000000 P 76bf52b0ac5147ffb2dac31e2b6f19ad [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 10:17:19.859265 3784 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 76bf52b0ac5147ffb2dac31e2b6f19ad [term 1 FOLLOWER]: Leader election won for term 1
I20260619 10:17:19.859372 3784 raft_consensus.cc:686] T 00000000000000000000000000000000 P 76bf52b0ac5147ffb2dac31e2b6f19ad [term 1 LEADER]: Becoming Leader. State: Replica: 76bf52b0ac5147ffb2dac31e2b6f19ad, State: Running, Role: LEADER
I20260619 10:17:19.859464 3779 sys_catalog.cc:564] T 00000000000000000000000000000000 P 76bf52b0ac5147ffb2dac31e2b6f19ad [sys.catalog]: configured and running, proceeding with master startup.
I20260619 10:17:19.859453 3784 consensus_queue.cc:237] T 00000000000000000000000000000000 P 76bf52b0ac5147ffb2dac31e2b6f19ad [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "76bf52b0ac5147ffb2dac31e2b6f19ad" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 38323 } }
I20260619 10:17:19.859746 3784 sys_catalog.cc:455] T 00000000000000000000000000000000 P 76bf52b0ac5147ffb2dac31e2b6f19ad [sys.catalog]: SysCatalogTable state changed. Reason: New leader 76bf52b0ac5147ffb2dac31e2b6f19ad. Latest consensus state: current_term: 1 leader_uuid: "76bf52b0ac5147ffb2dac31e2b6f19ad" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "76bf52b0ac5147ffb2dac31e2b6f19ad" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 38323 } } }
I20260619 10:17:19.859822 3784 sys_catalog.cc:458] T 00000000000000000000000000000000 P 76bf52b0ac5147ffb2dac31e2b6f19ad [sys.catalog]: This master's current role is: LEADER
I20260619 10:17:19.859757 3786 sys_catalog.cc:455] T 00000000000000000000000000000000 P 76bf52b0ac5147ffb2dac31e2b6f19ad [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "76bf52b0ac5147ffb2dac31e2b6f19ad" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "76bf52b0ac5147ffb2dac31e2b6f19ad" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 38323 } } }
I20260619 10:17:19.860010 3786 sys_catalog.cc:458] T 00000000000000000000000000000000 P 76bf52b0ac5147ffb2dac31e2b6f19ad [sys.catalog]: This master's current role is: LEADER
I20260619 10:17:19.860041 3789 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 10:17:19.860555 3789 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 10:17:19.862435 3789 catalog_manager.cc:1347] Generated new cluster ID: bc5e74bfdbd14a5983774bd9c4def101
I20260619 10:17:19.862488 3789 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Jun 19 10:17:19 dist-test-slave-6pt2 krb5kdc[3698](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864239, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.206.126@KRBTEST.COM
I20260619 10:17:19.868461 3783 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:19.857536 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42508 (local address 127.0.206.126:38323)
0619 10:17:19.857997 (+ 461us) server_negotiation.cc:206] Beginning negotiation
0619 10:17:19.858006 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 10:17:19.858041 (+ 35us) server_negotiation.cc:407] Connection header received
0619 10:17:19.858145 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:17:19.858150 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:17:19.858188 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:17:19.858374 (+ 186us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:17:19.859117 (+ 743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:19.860481 (+ 1364us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:19.861564 (+ 1083us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:19.862163 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:19.862298 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:19.864816 (+ 2518us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 10:17:19.864820 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 10:17:19.864820 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 10:17:19.864831 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0619 10:17:19.866870 (+ 2039us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:17:19.867261 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:17:19.867264 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:17:19.867264 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:17:19.867381 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:17:19.867612 (+ 231us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:17:19.867613 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:17:19.867614 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:17:19.867899 (+ 285us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 10:17:19.868069 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0619 10:17:19.868137 (+ 68us) server_negotiation.cc:299] Negotiation successful
0619 10:17:19.868267 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":401,"thread_start_us":352,"threads_started":1}
I20260619 10:17:19.868829 3789 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 10:17:19.869436 825 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 10:17:19.869515 3789 catalog_manager.cc:1504] Loading token signing keys...
I20260619 10:17:19.879014 3789 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 76bf52b0ac5147ffb2dac31e2b6f19ad: Generated new TSK 0
I20260619 10:17:19.879163 3789 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 10:17:19.888538 3783 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:19.885288 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42518 (local address 127.0.206.126:38323)
0619 10:17:19.885341 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0619 10:17:19.885343 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 10:17:19.885463 (+ 120us) server_negotiation.cc:407] Connection header received
0619 10:17:19.885535 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:17:19.885536 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:17:19.885546 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:17:19.885615 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:17:19.885959 (+ 344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:19.886438 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:19.887199 (+ 761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:19.887414 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:19.887496 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:19.888129 (+ 633us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 10:17:19.888130 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 10:17:19.888130 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 10:17:19.888317 (+ 187us) server_negotiation.cc:922] calling sasl_server_start()
0619 10:17:19.888329 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 10:17:19.888334 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 10:17:19.888392 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0619 10:17:19.888447 (+ 55us) server_negotiation.cc:299] Negotiation successful
0619 10:17:19.888468 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260619 10:17:19.890674 825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTU9OP6/build/release/bin/kudu with pid 3711
2026-06-19T10:17:19Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (1264 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781864224577058-825-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 10:17:19 dist-test-slave-6pt2 krb5kdc[3812](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 10:17:19 dist-test-slave-6pt2 krb5kdc[3812](info): set up 2 sockets
Jun 19 10:17:19 dist-test-slave-6pt2 krb5kdc[3812](info): commencing operation
krb5kdc: starting...
W20260619 10:17:20.914427 825 mini_kdc.cc:121] Time spent starting KDC: real 1.006s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781864224577058-825-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781864224577058-825-0/krb5kdc/test-user.keytab.
Jun 19 10:17:20 dist-test-slave-6pt2 krb5kdc[3812](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864240, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T10:17:20Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T10:17:20Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.206.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.206.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.206.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781864224577058-825-0/krb5kdc/kudu_127.0.206.126.keytab.
Entry for principal kudu/127.0.206.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781864224577058-825-0/krb5kdc/kudu_127.0.206.126.keytab.
I20260619 10:17:21.021013 825 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTU9OP6/build/release/bin/kudu
/tmp/dist-test-taskTU9OP6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781864224577058-825-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781864224577058-825-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781864224577058-825-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781864224577058-825-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.206.126:37089
--webserver_interface=127.0.206.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781864224577058-825-0/krb5kdc/kudu_127.0.206.126.keytab
--principal=kudu/127.0.206.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.206.84:43381
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.206.126:37089
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781864224577058-825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781864224577058-825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781864224577058-825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 10:17:21.098511 3825 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 10:17:21.098752 3825 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 10:17:21.098778 3825 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 10:17:21.099812 3825 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 10:17:21.099843 3825 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 10:17:21.099857 3825 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 10:17:21.099869 3825 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 10:17:21.099880 3825 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 10:17:21.101223 3825 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.206.84:43381
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781864224577058-825-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781864224577058-825-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.206.126:37089
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781864224577058-825-0/krb5kdc/kudu_127.0.206.126.keytab
--principal=kudu/127.0.206.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.206.126:37089
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781864224577058-825-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.206.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3825
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781864224577058-825-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 09:32:44 UTC on ccycloud.quasar-gbtldp.root.comops.site
build id 4455440
I20260619 10:17:21.101397 3825 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 10:17:21.101559 3825 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 10:17:21.106053 3830 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 10:17:21.106061 3833 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 10:17:21.106189 3831 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 10:17:21.106169 3825 server_base.cc:1034] running on GCE node
I20260619 10:17:21.106554 3825 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 10:17:21.106848 3825 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 10:17:21.108011 3825 hybrid_clock.cc:648] HybridClock initialized: now 1781864241107996 us; error 45 us; skew 500 ppm
Jun 19 10:17:21 dist-test-slave-6pt2 krb5kdc[3812](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864241, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 10:17:21.110349 3825 init.cc:377] Logged in from keytab as kudu/127.0.206.126@KRBTEST.COM (short username kudu)
I20260619 10:17:21.110817 3825 webserver.cc:460] Webserver started at http://127.0.206.126:43555/ using document root <none> and password file <none>
I20260619 10:17:21.110994 3825 fs_manager.cc:359] Metadata directory not provided
I20260619 10:17:21.111032 3825 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 10:17:21.111133 3825 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 10:17:21.112051 3825 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781864224577058-825-0/minicluster-data/master-0/data/instance:
uuid: "07b469aa19f448caa407e821bd78eb2b"
format_stamp: "Formatted at 2026-06-19 10:17:21 on dist-test-slave-6pt2"
server_key: "6f97f406a05dcbc14951cb88ffa2c16d"
server_key_iv: "a0ddd3e08a4e828a249ece75a7b1740b"
server_key_version: "encryptionkey@0"
I20260619 10:17:21.112334 3825 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781864224577058-825-0/minicluster-data/master-0/wal/instance:
uuid: "07b469aa19f448caa407e821bd78eb2b"
format_stamp: "Formatted at 2026-06-19 10:17:21 on dist-test-slave-6pt2"
server_key: "6f97f406a05dcbc14951cb88ffa2c16d"
server_key_iv: "a0ddd3e08a4e828a249ece75a7b1740b"
server_key_version: "encryptionkey@0"
I20260619 10:17:21.113511 3825 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260619 10:17:21.114117 3840 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 10:17:21.114284 3825 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260619 10:17:21.114369 3825 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781864224577058-825-0/minicluster-data/master-0/data,/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781864224577058-825-0/minicluster-data/master-0/wal
uuid: "07b469aa19f448caa407e821bd78eb2b"
format_stamp: "Formatted at 2026-06-19 10:17:21 on dist-test-slave-6pt2"
server_key: "6f97f406a05dcbc14951cb88ffa2c16d"
server_key_iv: "a0ddd3e08a4e828a249ece75a7b1740b"
server_key_version: "encryptionkey@0"
I20260619 10:17:21.114439 3825 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781864224577058-825-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781864224577058-825-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781864224577058-825-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 10:17:21.129587 3825 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 10:17:21.131413 3825 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 10:17:21.131564 3825 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 10:17:21.135160 3825 rpc_server.cc:307] RPC server started. Bound to: 127.0.206.126:37089
I20260619 10:17:21.135190 3892 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.206.126:37089 every 8 connection(s)
I20260619 10:17:21.135521 3825 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781864224577058-825-0/minicluster-data/master-0/data/info.pb
I20260619 10:17:21.136446 3893 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 10:17:21.138571 3893 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 07b469aa19f448caa407e821bd78eb2b: Bootstrap starting.
I20260619 10:17:21.139184 3893 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 07b469aa19f448caa407e821bd78eb2b: Neither blocks nor log segments found. Creating new log.
I20260619 10:17:21.139456 3893 log.cc:824] T 00000000000000000000000000000000 P 07b469aa19f448caa407e821bd78eb2b: Log is configured to *not* fsync() on all Append() calls
I20260619 10:17:21.140101 3893 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 07b469aa19f448caa407e821bd78eb2b: No bootstrap required, opened a new log
I20260619 10:17:21.141623 3893 raft_consensus.cc:348] T 00000000000000000000000000000000 P 07b469aa19f448caa407e821bd78eb2b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "07b469aa19f448caa407e821bd78eb2b" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 37089 } }
I20260619 10:17:21.141780 3893 raft_consensus.cc:374] T 00000000000000000000000000000000 P 07b469aa19f448caa407e821bd78eb2b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 10:17:21.141830 3893 raft_consensus.cc:729] T 00000000000000000000000000000000 P 07b469aa19f448caa407e821bd78eb2b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 07b469aa19f448caa407e821bd78eb2b, State: Initialized, Role: FOLLOWER
I20260619 10:17:21.141992 3893 consensus_queue.cc:260] T 00000000000000000000000000000000 P 07b469aa19f448caa407e821bd78eb2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "07b469aa19f448caa407e821bd78eb2b" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 37089 } }
I20260619 10:17:21.142074 3893 raft_consensus.cc:388] T 00000000000000000000000000000000 P 07b469aa19f448caa407e821bd78eb2b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 10:17:21.142113 3893 raft_consensus.cc:482] T 00000000000000000000000000000000 P 07b469aa19f448caa407e821bd78eb2b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 10:17:21.142163 3893 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 07b469aa19f448caa407e821bd78eb2b [term 0 FOLLOWER]: Advancing to term 1
I20260619 10:17:21.143033 3893 raft_consensus.cc:504] T 00000000000000000000000000000000 P 07b469aa19f448caa407e821bd78eb2b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "07b469aa19f448caa407e821bd78eb2b" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 37089 } }
I20260619 10:17:21.143144 3893 leader_election.cc:302] T 00000000000000000000000000000000 P 07b469aa19f448caa407e821bd78eb2b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 07b469aa19f448caa407e821bd78eb2b; no voters:
I20260619 10:17:21.143322 3893 leader_election.cc:288] T 00000000000000000000000000000000 P 07b469aa19f448caa407e821bd78eb2b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 10:17:21.143357 3896 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 07b469aa19f448caa407e821bd78eb2b [term 1 FOLLOWER]: Leader election won for term 1
I20260619 10:17:21.143524 3896 raft_consensus.cc:686] T 00000000000000000000000000000000 P 07b469aa19f448caa407e821bd78eb2b [term 1 LEADER]: Becoming Leader. State: Replica: 07b469aa19f448caa407e821bd78eb2b, State: Running, Role: LEADER
I20260619 10:17:21.143573 3893 sys_catalog.cc:564] T 00000000000000000000000000000000 P 07b469aa19f448caa407e821bd78eb2b [sys.catalog]: configured and running, proceeding with master startup.
I20260619 10:17:21.143682 3896 consensus_queue.cc:237] T 00000000000000000000000000000000 P 07b469aa19f448caa407e821bd78eb2b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "07b469aa19f448caa407e821bd78eb2b" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 37089 } }
I20260619 10:17:21.144017 3897 sys_catalog.cc:455] T 00000000000000000000000000000000 P 07b469aa19f448caa407e821bd78eb2b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "07b469aa19f448caa407e821bd78eb2b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "07b469aa19f448caa407e821bd78eb2b" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 37089 } } }
I20260619 10:17:21.144129 3897 sys_catalog.cc:458] T 00000000000000000000000000000000 P 07b469aa19f448caa407e821bd78eb2b [sys.catalog]: This master's current role is: LEADER
I20260619 10:17:21.144019 3898 sys_catalog.cc:455] T 00000000000000000000000000000000 P 07b469aa19f448caa407e821bd78eb2b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 07b469aa19f448caa407e821bd78eb2b. Latest consensus state: current_term: 1 leader_uuid: "07b469aa19f448caa407e821bd78eb2b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "07b469aa19f448caa407e821bd78eb2b" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 37089 } } }
I20260619 10:17:21.144259 3898 sys_catalog.cc:458] T 00000000000000000000000000000000 P 07b469aa19f448caa407e821bd78eb2b [sys.catalog]: This master's current role is: LEADER
I20260619 10:17:21.144577 3905 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 10:17:21.144984 3905 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 10:17:21.145197 825 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTU9OP6/build/release/bin/kudu as pid 3825
I20260619 10:17:21.145298 825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781864224577058-825-0/minicluster-data/master-0/wal/instance
I20260619 10:17:21.145475 825 external_mini_cluster.cc:1428] Setting key 45bdde2c8a77e1eb637be1a2d588eb47
I20260619 10:17:21.146723 3905 catalog_manager.cc:1347] Generated new cluster ID: 8ce38ba06dbb4c03a907a9b889d1a1ff
I20260619 10:17:21.146771 3905 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 10:17:21.151458 3905 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 10:17:21.151952 3905 catalog_manager.cc:1504] Loading token signing keys...
Jun 19 10:17:21 dist-test-slave-6pt2 krb5kdc[3812](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864240, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.206.126@KRBTEST.COM
I20260619 10:17:21.155721 3905 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 07b469aa19f448caa407e821bd78eb2b: Generated new TSK 0
I20260619 10:17:21.155871 3905 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 10:17:21.157011 3915 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:21.145883 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50338 (local address 127.0.206.126:37089)
0619 10:17:21.146090 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0619 10:17:21.146099 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 10:17:21.146302 (+ 203us) server_negotiation.cc:407] Connection header received
0619 10:17:21.146448 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:17:21.146455 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:17:21.146511 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:17:21.146708 (+ 197us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:17:21.147575 (+ 867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:21.148984 (+ 1409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:21.150062 (+ 1078us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:21.150689 (+ 627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:21.150785 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:21.153642 (+ 2857us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 10:17:21.153645 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 10:17:21.153646 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 10:17:21.153657 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0619 10:17:21.155635 (+ 1978us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:17:21.156019 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:17:21.156022 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:17:21.156022 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:17:21.156115 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:17:21.156327 (+ 212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:17:21.156327 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:17:21.156328 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:17:21.156521 (+ 193us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 10:17:21.156612 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0619 10:17:21.156765 (+ 153us) server_negotiation.cc:299] Negotiation successful
0619 10:17:21.156870 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":151,"thread_start_us":86,"threads_started":1}
WARNING: no policy specified for kudu/127.0.206.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.206.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.206.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781864224577058-825-0/krb5kdc/kudu_127.0.206.65.keytab.
Entry for principal kudu/127.0.206.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781864224577058-825-0/krb5kdc/kudu_127.0.206.65.keytab.
I20260619 10:17:21.178591 825 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTU9OP6/build/release/bin/kudu
/tmp/dist-test-taskTU9OP6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781864224577058-825-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781864224577058-825-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781864224577058-825-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781864224577058-825-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.206.65:0
--local_ip_for_outbound_sockets=127.0.206.65
--webserver_interface=127.0.206.65
--webserver_port=0
--tserver_master_addrs=127.0.206.126:37089
--keytab_file=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781864224577058-825-0/krb5kdc/kudu_127.0.206.65.keytab
--principal=kudu/127.0.206.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.206.84:43381
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781864224577058-825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781864224577058-825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781864224577058-825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 10:17:21.258001 3919 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 10:17:21.258216 3919 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 10:17:21.258253 3919 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 10:17:21.259473 3919 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 10:17:21.259538 3919 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 10:17:21.259590 3919 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.206.65
I20260619 10:17:21.261032 3919 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.206.84:43381
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781864224577058-825-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781864224577058-825-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781864224577058-825-0/krb5kdc/kudu_127.0.206.65.keytab
--principal=kudu/127.0.206.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.206.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781864224577058-825-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.206.65
--webserver_port=0
--tserver_master_addrs=127.0.206.126:37089
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3919
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.206.65
--log_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781864224577058-825-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 09:32:44 UTC on ccycloud.quasar-gbtldp.root.comops.site
build id 4455440
I20260619 10:17:21.261286 3919 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 10:17:21.261519 3919 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 10:17:21.266446 3924 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 10:17:21.266613 3919 server_base.cc:1034] running on GCE node
W20260619 10:17:21.266450 3927 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 10:17:21.266722 3925 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 10:17:21.266932 3919 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 10:17:21.267213 3919 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 10:17:21.268371 3919 hybrid_clock.cc:648] HybridClock initialized: now 1781864241268361 us; error 30 us; skew 500 ppm
Jun 19 10:17:21 dist-test-slave-6pt2 krb5kdc[3812](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864241, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 10:17:21.270877 3919 init.cc:377] Logged in from keytab as kudu/127.0.206.65@KRBTEST.COM (short username kudu)
I20260619 10:17:21.271267 3919 webserver.cc:460] Webserver started at http://127.0.206.65:37173/ using document root <none> and password file <none>
I20260619 10:17:21.271494 3919 fs_manager.cc:359] Metadata directory not provided
I20260619 10:17:21.271559 3919 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 10:17:21.271713 3919 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 10:17:21.272711 3919 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781864224577058-825-0/minicluster-data/ts-0/data/instance:
uuid: "06105c4be77744d6a54e99fbc9e75afe"
format_stamp: "Formatted at 2026-06-19 10:17:21 on dist-test-slave-6pt2"
server_key: "c19f8b038976dbe1f52e6d1b66279289"
server_key_iv: "46240442743fd6609a5e192b70ab3164"
server_key_version: "encryptionkey@0"
I20260619 10:17:21.273077 3919 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781864224577058-825-0/minicluster-data/ts-0/wal/instance:
uuid: "06105c4be77744d6a54e99fbc9e75afe"
format_stamp: "Formatted at 2026-06-19 10:17:21 on dist-test-slave-6pt2"
server_key: "c19f8b038976dbe1f52e6d1b66279289"
server_key_iv: "46240442743fd6609a5e192b70ab3164"
server_key_version: "encryptionkey@0"
I20260619 10:17:21.274317 3919 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.002s
I20260619 10:17:21.275072 3934 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 10:17:21.275242 3919 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260619 10:17:21.275315 3919 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781864224577058-825-0/minicluster-data/ts-0/data,/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781864224577058-825-0/minicluster-data/ts-0/wal
uuid: "06105c4be77744d6a54e99fbc9e75afe"
format_stamp: "Formatted at 2026-06-19 10:17:21 on dist-test-slave-6pt2"
server_key: "c19f8b038976dbe1f52e6d1b66279289"
server_key_iv: "46240442743fd6609a5e192b70ab3164"
server_key_version: "encryptionkey@0"
I20260619 10:17:21.275399 3919 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781864224577058-825-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781864224577058-825-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781864224577058-825-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 10:17:21.287357 3919 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 10:17:21.289433 3919 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 10:17:21.289603 3919 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 10:17:21.289812 3919 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 10:17:21.290107 3919 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 10:17:21.290160 3919 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 10:17:21.290199 3919 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 10:17:21.290227 3919 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 10:17:21.296063 3919 rpc_server.cc:307] RPC server started. Bound to: 127.0.206.65:43221
I20260619 10:17:21.296110 4047 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.206.65:43221 every 8 connection(s)
I20260619 10:17:21.296381 3919 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781864224577058-825-0/minicluster-data/ts-0/data/info.pb
Jun 19 10:17:21 dist-test-slave-6pt2 krb5kdc[3812](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864241, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.65@KRBTEST.COM for kudu/127.0.206.126@KRBTEST.COM
I20260619 10:17:21.303023 825 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTU9OP6/build/release/bin/kudu as pid 3919
I20260619 10:17:21.303134 825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781864224577058-825-0/minicluster-data/ts-0/wal/instance
I20260619 10:17:21.303344 825 external_mini_cluster.cc:1428] Setting key ebb5a129a35cf1cbdf0447314c0db8a3
I20260619 10:17:21.304698 3915 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:21.296992 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.65:36295 (local address 127.0.206.126:37089)
0619 10:17:21.297031 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0619 10:17:21.297033 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 10:17:21.297209 (+ 176us) server_negotiation.cc:407] Connection header received
0619 10:17:21.297617 (+ 408us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:17:21.297618 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:17:21.297626 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:17:21.297682 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:17:21.298492 (+ 810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:21.299043 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:21.299924 (+ 881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:21.300105 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:21.300154 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:21.301991 (+ 1837us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 10:17:21.301993 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 10:17:21.301993 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 10:17:21.301998 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0619 10:17:21.303357 (+ 1359us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:17:21.303795 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:17:21.303796 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:17:21.303797 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:17:21.303854 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:17:21.304117 (+ 263us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:17:21.304117 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:17:21.304118 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:17:21.304218 (+ 100us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 10:17:21.304272 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0619 10:17:21.304576 (+ 304us) server_negotiation.cc:299] Negotiation successful
0619 10:17:21.304639 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260619 10:17:21.305027 4050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:21.296944 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:37089 (local address 127.0.206.65:36295)
0619 10:17:21.297086 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0619 10:17:21.297133 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0619 10:17:21.297545 (+ 412us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 10:17:21.297728 (+ 183us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 10:17:21.297730 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 10:17:21.298252 (+ 522us) client_negotiation.cc:189] Negotiated authn=SASL
0619 10:17:21.298420 (+ 168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:21.298425 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:21.299123 (+ 698us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:21.299123 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:21.299853 (+ 730us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:21.299855 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:21.300171 (+ 316us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:21.300171 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:21.300262 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:21.300654 (+ 392us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 10:17:21.300662 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 10:17:21.301904 (+ 1242us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 10:17:21.303450 (+ 1546us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 10:17:21.303456 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 10:17:21.303458 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 10:17:21.303718 (+ 260us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 10:17:21.303925 (+ 207us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 10:17:21.303927 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 10:17:21.303927 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 10:17:21.304048 (+ 121us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 10:17:21.304285 (+ 237us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 10:17:21.304291 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 10:17:21.304446 (+ 155us) client_negotiation.cc:769] Sending connection context
0619 10:17:21.304557 (+ 111us) client_negotiation.cc:240] Negotiation successful
0619 10:17:21.304721 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":85,"threads_started":1}
I20260619 10:17:21.305500 4048 heartbeater.cc:347] Connected to a master server at 127.0.206.126:37089
I20260619 10:17:21.305608 4048 heartbeater.cc:464] Registering TS with master...
I20260619 10:17:21.305765 4048 heartbeater.cc:511] Master 127.0.206.126:37089 requested a full tablet report, sending...
I20260619 10:17:21.306232 3857 ts_manager.cc:194] Registered new tserver with Master: 06105c4be77744d6a54e99fbc9e75afe (127.0.206.65:43221)
I20260619 10:17:21.306896 3857 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.206.65@KRBTEST.COM'} at 127.0.206.65:36295
WARNING: no policy specified for kudu/127.0.206.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.206.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.206.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781864224577058-825-0/krb5kdc/kudu_127.0.206.66.keytab.
Entry for principal kudu/127.0.206.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781864224577058-825-0/krb5kdc/kudu_127.0.206.66.keytab.
I20260619 10:17:21.326767 825 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTU9OP6/build/release/bin/kudu
/tmp/dist-test-taskTU9OP6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781864224577058-825-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781864224577058-825-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781864224577058-825-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781864224577058-825-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.206.66:0
--local_ip_for_outbound_sockets=127.0.206.66
--webserver_interface=127.0.206.66
--webserver_port=0
--tserver_master_addrs=127.0.206.126:37089
--keytab_file=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781864224577058-825-0/krb5kdc/kudu_127.0.206.66.keytab
--principal=kudu/127.0.206.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.206.84:43381
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781864224577058-825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781864224577058-825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781864224577058-825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 10:17:21.402734 4053 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 10:17:21.402916 4053 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 10:17:21.402938 4053 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 10:17:21.403973 4053 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 10:17:21.404007 4053 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 10:17:21.404033 4053 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.206.66
I20260619 10:17:21.405365 4053 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.206.84:43381
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781864224577058-825-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781864224577058-825-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781864224577058-825-0/krb5kdc/kudu_127.0.206.66.keytab
--principal=kudu/127.0.206.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.206.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781864224577058-825-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.206.66
--webserver_port=0
--tserver_master_addrs=127.0.206.126:37089
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4053
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.206.66
--log_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781864224577058-825-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 09:32:44 UTC on ccycloud.quasar-gbtldp.root.comops.site
build id 4455440
I20260619 10:17:21.405524 4053 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 10:17:21.405676 4053 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 10:17:21.410452 4059 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 10:17:21.410444 4061 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 10:17:21.410545 4053 server_base.cc:1034] running on GCE node
W20260619 10:17:21.410543 4058 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 10:17:21.410904 4053 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 10:17:21.411118 4053 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 10:17:21.412273 4053 hybrid_clock.cc:648] HybridClock initialized: now 1781864241412250 us; error 39 us; skew 500 ppm
Jun 19 10:17:21 dist-test-slave-6pt2 krb5kdc[3812](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864241, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 10:17:21.414585 4053 init.cc:377] Logged in from keytab as kudu/127.0.206.66@KRBTEST.COM (short username kudu)
I20260619 10:17:21.415091 4053 webserver.cc:460] Webserver started at http://127.0.206.66:42485/ using document root <none> and password file <none>
I20260619 10:17:21.415270 4053 fs_manager.cc:359] Metadata directory not provided
I20260619 10:17:21.415306 4053 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 10:17:21.415390 4053 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 10:17:21.416312 4053 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781864224577058-825-0/minicluster-data/ts-1/data/instance:
uuid: "ee52047102914668a7a0d84a818c216a"
format_stamp: "Formatted at 2026-06-19 10:17:21 on dist-test-slave-6pt2"
server_key: "da8009458242f20dd45d3370290bf3b5"
server_key_iv: "6d2fa3d4f8207f51214105a3f4a570aa"
server_key_version: "encryptionkey@0"
I20260619 10:17:21.416646 4053 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781864224577058-825-0/minicluster-data/ts-1/wal/instance:
uuid: "ee52047102914668a7a0d84a818c216a"
format_stamp: "Formatted at 2026-06-19 10:17:21 on dist-test-slave-6pt2"
server_key: "da8009458242f20dd45d3370290bf3b5"
server_key_iv: "6d2fa3d4f8207f51214105a3f4a570aa"
server_key_version: "encryptionkey@0"
I20260619 10:17:21.417956 4053 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260619 10:17:21.418553 4068 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 10:17:21.418748 4053 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260619 10:17:21.418833 4053 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781864224577058-825-0/minicluster-data/ts-1/data,/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781864224577058-825-0/minicluster-data/ts-1/wal
uuid: "ee52047102914668a7a0d84a818c216a"
format_stamp: "Formatted at 2026-06-19 10:17:21 on dist-test-slave-6pt2"
server_key: "da8009458242f20dd45d3370290bf3b5"
server_key_iv: "6d2fa3d4f8207f51214105a3f4a570aa"
server_key_version: "encryptionkey@0"
I20260619 10:17:21.418910 4053 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781864224577058-825-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781864224577058-825-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781864224577058-825-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 10:17:21.437016 4053 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 10:17:21.438802 4053 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 10:17:21.438951 4053 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 10:17:21.439186 4053 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 10:17:21.439495 4053 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 10:17:21.439559 4053 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 10:17:21.439611 4053 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 10:17:21.439639 4053 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 10:17:21.445672 4053 rpc_server.cc:307] RPC server started. Bound to: 127.0.206.66:41121
I20260619 10:17:21.445712 4181 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.206.66:41121 every 8 connection(s)
I20260619 10:17:21.446017 4053 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781864224577058-825-0/minicluster-data/ts-1/data/info.pb
I20260619 10:17:21.450768 825 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTU9OP6/build/release/bin/kudu as pid 4053
I20260619 10:17:21.450877 825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781864224577058-825-0/minicluster-data/ts-1/wal/instance
I20260619 10:17:21.451069 825 external_mini_cluster.cc:1428] Setting key f0aa236fa868d827fe77195a0321d99f
Jun 19 10:17:21 dist-test-slave-6pt2 krb5kdc[3812](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864241, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.66@KRBTEST.COM for kudu/127.0.206.126@KRBTEST.COM
I20260619 10:17:21.455484 3915 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:21.446729 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.66:47037 (local address 127.0.206.126:37089)
0619 10:17:21.446821 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0619 10:17:21.446823 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 10:17:21.446998 (+ 175us) server_negotiation.cc:407] Connection header received
0619 10:17:21.447520 (+ 522us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:17:21.447521 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:17:21.447533 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:17:21.447584 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:17:21.448508 (+ 924us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:21.449000 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:21.449968 (+ 968us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:21.450174 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:21.450247 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:21.452613 (+ 2366us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 10:17:21.452614 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 10:17:21.452614 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 10:17:21.452622 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 10:17:21.454029 (+ 1407us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:17:21.454509 (+ 480us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:17:21.454509 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:17:21.454510 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:17:21.454558 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:17:21.454839 (+ 281us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:17:21.454840 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:17:21.454840 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:17:21.454952 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 10:17:21.455006 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0619 10:17:21.455353 (+ 347us) server_negotiation.cc:299] Negotiation successful
0619 10:17:21.455423 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260619 10:17:21.455721 4184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:21.446654 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:37089 (local address 127.0.206.66:47037)
0619 10:17:21.446863 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0619 10:17:21.446911 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0619 10:17:21.447408 (+ 497us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 10:17:21.447649 (+ 241us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 10:17:21.447653 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 10:17:21.448206 (+ 553us) client_negotiation.cc:189] Negotiated authn=SASL
0619 10:17:21.448400 (+ 194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:21.448405 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:21.449098 (+ 693us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:21.449099 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:21.449871 (+ 772us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:21.449873 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:21.450249 (+ 376us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:21.450250 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:21.450406 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:21.450968 (+ 562us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 10:17:21.450979 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0619 10:17:21.452527 (+ 1548us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 10:17:21.454109 (+ 1582us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 10:17:21.454115 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 10:17:21.454117 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 10:17:21.454428 (+ 311us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 10:17:21.454634 (+ 206us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 10:17:21.454635 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 10:17:21.454635 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 10:17:21.454773 (+ 138us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 10:17:21.455027 (+ 254us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 10:17:21.455033 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 10:17:21.455197 (+ 164us) client_negotiation.cc:769] Sending connection context
0619 10:17:21.455312 (+ 115us) client_negotiation.cc:240] Negotiation successful
0619 10:17:21.455484 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":101,"threads_started":1}
I20260619 10:17:21.456249 4182 heartbeater.cc:347] Connected to a master server at 127.0.206.126:37089
I20260619 10:17:21.456378 4182 heartbeater.cc:464] Registering TS with master...
I20260619 10:17:21.456548 4182 heartbeater.cc:511] Master 127.0.206.126:37089 requested a full tablet report, sending...
I20260619 10:17:21.456947 3857 ts_manager.cc:194] Registered new tserver with Master: ee52047102914668a7a0d84a818c216a (127.0.206.66:41121)
I20260619 10:17:21.457319 3857 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.206.66@KRBTEST.COM'} at 127.0.206.66:47037
WARNING: no policy specified for kudu/127.0.206.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.206.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.206.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781864224577058-825-0/krb5kdc/kudu_127.0.206.67.keytab.
Entry for principal kudu/127.0.206.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781864224577058-825-0/krb5kdc/kudu_127.0.206.67.keytab.
I20260619 10:17:21.473120 825 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTU9OP6/build/release/bin/kudu
/tmp/dist-test-taskTU9OP6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781864224577058-825-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781864224577058-825-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781864224577058-825-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781864224577058-825-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.206.67:0
--local_ip_for_outbound_sockets=127.0.206.67
--webserver_interface=127.0.206.67
--webserver_port=0
--tserver_master_addrs=127.0.206.126:37089
--keytab_file=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781864224577058-825-0/krb5kdc/kudu_127.0.206.67.keytab
--principal=kudu/127.0.206.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.206.84:43381
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781864224577058-825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781864224577058-825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781864224577058-825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 10:17:21.550858 4187 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 10:17:21.551043 4187 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 10:17:21.551065 4187 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 10:17:21.552100 4187 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 10:17:21.552134 4187 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 10:17:21.552160 4187 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.206.67
I20260619 10:17:21.553474 4187 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.206.84:43381
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781864224577058-825-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781864224577058-825-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781864224577058-825-0/krb5kdc/kudu_127.0.206.67.keytab
--principal=kudu/127.0.206.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.206.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781864224577058-825-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.206.67
--webserver_port=0
--tserver_master_addrs=127.0.206.126:37089
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4187
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.206.67
--log_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781864224577058-825-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 09:32:44 UTC on ccycloud.quasar-gbtldp.root.comops.site
build id 4455440
I20260619 10:17:21.553646 4187 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 10:17:21.553797 4187 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 10:17:21.558367 4195 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 10:17:21.558419 4193 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 10:17:21.558390 4192 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 10:17:21.558974 4187 server_base.cc:1034] running on GCE node
I20260619 10:17:21.559171 4187 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 10:17:21.559381 4187 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 10:17:21.560546 4187 hybrid_clock.cc:648] HybridClock initialized: now 1781864241560534 us; error 27 us; skew 500 ppm
Jun 19 10:17:21 dist-test-slave-6pt2 krb5kdc[3812](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864241, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 10:17:21.563294 4187 init.cc:377] Logged in from keytab as kudu/127.0.206.67@KRBTEST.COM (short username kudu)
I20260619 10:17:21.563778 4187 webserver.cc:460] Webserver started at http://127.0.206.67:45751/ using document root <none> and password file <none>
I20260619 10:17:21.563979 4187 fs_manager.cc:359] Metadata directory not provided
I20260619 10:17:21.564023 4187 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 10:17:21.564114 4187 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 10:17:21.565160 4187 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781864224577058-825-0/minicluster-data/ts-2/data/instance:
uuid: "de2168a537bb4ded98c09d32228ac8b5"
format_stamp: "Formatted at 2026-06-19 10:17:21 on dist-test-slave-6pt2"
server_key: "a77c3cbce31a2c096086dc1a18219ef3"
server_key_iv: "a831479dd828d4408a2702bb98b080bb"
server_key_version: "encryptionkey@0"
I20260619 10:17:21.565485 4187 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781864224577058-825-0/minicluster-data/ts-2/wal/instance:
uuid: "de2168a537bb4ded98c09d32228ac8b5"
format_stamp: "Formatted at 2026-06-19 10:17:21 on dist-test-slave-6pt2"
server_key: "a77c3cbce31a2c096086dc1a18219ef3"
server_key_iv: "a831479dd828d4408a2702bb98b080bb"
server_key_version: "encryptionkey@0"
I20260619 10:17:21.566818 4187 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 10:17:21.567503 4202 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 10:17:21.567672 4187 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 10:17:21.567767 4187 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781864224577058-825-0/minicluster-data/ts-2/data,/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781864224577058-825-0/minicluster-data/ts-2/wal
uuid: "de2168a537bb4ded98c09d32228ac8b5"
format_stamp: "Formatted at 2026-06-19 10:17:21 on dist-test-slave-6pt2"
server_key: "a77c3cbce31a2c096086dc1a18219ef3"
server_key_iv: "a831479dd828d4408a2702bb98b080bb"
server_key_version: "encryptionkey@0"
I20260619 10:17:21.567848 4187 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781864224577058-825-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781864224577058-825-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781864224577058-825-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 10:17:21.580497 4187 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 10:17:21.582930 4187 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 10:17:21.583057 4187 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 10:17:21.583228 4187 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 10:17:21.583482 4187 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 10:17:21.583527 4187 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 10:17:21.583552 4187 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 10:17:21.583568 4187 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 10:17:21.590298 4187 rpc_server.cc:307] RPC server started. Bound to: 127.0.206.67:40177
I20260619 10:17:21.590394 4315 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.206.67:40177 every 8 connection(s)
I20260619 10:17:21.590734 4187 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781864224577058-825-0/minicluster-data/ts-2/data/info.pb
Jun 19 10:17:21 dist-test-slave-6pt2 krb5kdc[3812](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864241, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.67@KRBTEST.COM for kudu/127.0.206.126@KRBTEST.COM
I20260619 10:17:21.597792 825 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTU9OP6/build/release/bin/kudu as pid 4187
I20260619 10:17:21.597898 825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781864224577058-825-0/minicluster-data/ts-2/wal/instance
I20260619 10:17:21.598119 825 external_mini_cluster.cc:1428] Setting key 8d561696c93006234aacf630320bb4d9
I20260619 10:17:21.599658 3915 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:21.591430 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.67:51973 (local address 127.0.206.126:37089)
0619 10:17:21.591496 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0619 10:17:21.591499 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 10:17:21.591733 (+ 234us) server_negotiation.cc:407] Connection header received
0619 10:17:21.592180 (+ 447us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:17:21.592181 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:17:21.592194 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:17:21.592269 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:17:21.593247 (+ 978us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:21.593734 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:21.594699 (+ 965us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:21.594879 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:21.594931 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:21.596781 (+ 1850us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 10:17:21.596783 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 10:17:21.596783 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 10:17:21.596789 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 10:17:21.598354 (+ 1565us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:17:21.598870 (+ 516us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:17:21.598871 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:17:21.598872 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:17:21.598931 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:17:21.599207 (+ 276us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:17:21.599207 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:17:21.599208 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:17:21.599330 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 10:17:21.599380 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0619 10:17:21.599549 (+ 169us) server_negotiation.cc:299] Negotiation successful
0619 10:17:21.599600 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260619 10:17:21.599875 4318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:21.591394 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:37089 (local address 127.0.206.67:51973)
0619 10:17:21.591603 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0619 10:17:21.591642 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0619 10:17:21.592086 (+ 444us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 10:17:21.592355 (+ 269us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 10:17:21.592359 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 10:17:21.592990 (+ 631us) client_negotiation.cc:189] Negotiated authn=SASL
0619 10:17:21.593163 (+ 173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:21.593168 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:21.593810 (+ 642us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:21.593811 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:21.594559 (+ 748us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:21.594561 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:21.594946 (+ 385us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:21.594946 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:21.595044 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:21.595415 (+ 371us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 10:17:21.595422 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0619 10:17:21.596708 (+ 1286us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 10:17:21.598447 (+ 1739us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 10:17:21.598452 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 10:17:21.598455 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 10:17:21.598778 (+ 323us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 10:17:21.598997 (+ 219us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 10:17:21.598998 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 10:17:21.598998 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 10:17:21.599135 (+ 137us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 10:17:21.599372 (+ 237us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 10:17:21.599375 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 10:17:21.599468 (+ 93us) client_negotiation.cc:769] Sending connection context
0619 10:17:21.599537 (+ 69us) client_negotiation.cc:240] Negotiation successful
0619 10:17:21.599642 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":171,"thread_start_us":79,"threads_started":1}
I20260619 10:17:21.600308 4316 heartbeater.cc:347] Connected to a master server at 127.0.206.126:37089
I20260619 10:17:21.600397 4316 heartbeater.cc:464] Registering TS with master...
I20260619 10:17:21.600569 4316 heartbeater.cc:511] Master 127.0.206.126:37089 requested a full tablet report, sending...
I20260619 10:17:21.600922 3857 ts_manager.cc:194] Registered new tserver with Master: de2168a537bb4ded98c09d32228ac8b5 (127.0.206.67:40177)
I20260619 10:17:21.601243 3857 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.206.67@KRBTEST.COM'} at 127.0.206.67:51973
I20260619 10:17:21.609926 825 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Jun 19 10:17:21 dist-test-slave-6pt2 krb5kdc[3812](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864241, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Jun 19 10:17:21 dist-test-slave-6pt2 krb5kdc[3812](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864241, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.0.206.126@KRBTEST.COM
I20260619 10:17:21.631405 3915 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:21.624612 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33978 (local address 127.0.206.126:37089)
0619 10:17:21.624680 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0619 10:17:21.624682 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 10:17:21.624748 (+ 66us) server_negotiation.cc:407] Connection header received
0619 10:17:21.624781 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:17:21.624782 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:17:21.624790 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:17:21.624839 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:17:21.625308 (+ 469us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:21.625753 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:21.626449 (+ 696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:21.626685 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:21.626749 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:21.628900 (+ 2151us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 10:17:21.628902 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 10:17:21.628902 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 10:17:21.628908 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 10:17:21.630404 (+ 1496us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:17:21.630757 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:17:21.630758 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:17:21.630758 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:17:21.630807 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:17:21.630950 (+ 143us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:17:21.630951 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:17:21.630951 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:17:21.631074 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 10:17:21.631141 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0619 10:17:21.631224 (+ 83us) server_negotiation.cc:299] Negotiation successful
0619 10:17:21.631313 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
W20260619 10:17:21.631655 3857 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:33978
I20260619 10:17:21.632169 825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTU9OP6/build/release/bin/kudu with pid 3919
I20260619 10:17:21.636880 825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTU9OP6/build/release/bin/kudu with pid 4053
I20260619 10:17:21.641100 825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTU9OP6/build/release/bin/kudu with pid 4187
I20260619 10:17:21.645179 825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTU9OP6/build/release/bin/kudu with pid 3825
2026-06-19T10:17:21Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (1756 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781864224577058-825-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 10:17:21 dist-test-slave-6pt2 krb5kdc[4328](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 10:17:21 dist-test-slave-6pt2 krb5kdc[4328](info): set up 2 sockets
Jun 19 10:17:21 dist-test-slave-6pt2 krb5kdc[4328](info): commencing operation
krb5kdc: starting...
W20260619 10:17:22.690672 825 mini_kdc.cc:121] Time spent starting KDC: real 1.026s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781864224577058-825-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781864224577058-825-0/krb5kdc/test-user.keytab.
Jun 19 10:17:22 dist-test-slave-6pt2 krb5kdc[4328](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864242, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T10:17:22Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T10:17:22Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.206.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.206.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.206.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781864224577058-825-0/krb5kdc/kudu_127.0.206.126.keytab.
Entry for principal kudu/127.0.206.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781864224577058-825-0/krb5kdc/kudu_127.0.206.126.keytab.
I20260619 10:17:22.839589 825 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTU9OP6/build/release/bin/kudu
/tmp/dist-test-taskTU9OP6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781864224577058-825-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781864224577058-825-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781864224577058-825-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781864224577058-825-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.206.126:34083
--webserver_interface=127.0.206.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781864224577058-825-0/krb5kdc/kudu_127.0.206.126.keytab
--principal=kudu/127.0.206.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.206.84:40891
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.206.126:34083
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781864224577058-825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781864224577058-825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781864224577058-825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 10:17:22.918551 4341 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 10:17:22.918880 4341 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 10:17:22.918919 4341 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 10:17:22.920099 4341 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 10:17:22.920174 4341 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 10:17:22.920197 4341 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 10:17:22.920217 4341 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 10:17:22.920248 4341 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 10:17:22.921725 4341 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.206.84:40891
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781864224577058-825-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781864224577058-825-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.206.126:34083
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781864224577058-825-0/krb5kdc/kudu_127.0.206.126.keytab
--principal=kudu/127.0.206.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.206.126:34083
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781864224577058-825-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.206.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4341
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781864224577058-825-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 09:32:44 UTC on ccycloud.quasar-gbtldp.root.comops.site
build id 4455440
I20260619 10:17:22.922009 4341 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 10:17:22.922235 4341 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 10:17:22.927089 4349 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 10:17:22.927160 4347 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 10:17:22.927122 4346 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 10:17:22.927384 4341 server_base.cc:1034] running on GCE node
I20260619 10:17:22.927623 4341 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 10:17:22.927910 4341 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 10:17:22.929073 4341 hybrid_clock.cc:648] HybridClock initialized: now 1781864242929068 us; error 35 us; skew 500 ppm
Jun 19 10:17:22 dist-test-slave-6pt2 krb5kdc[4328](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864242, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 10:17:22.935099 4341 init.cc:377] Logged in from keytab as kudu/127.0.206.126@KRBTEST.COM (short username kudu)
I20260619 10:17:22.935495 4341 webserver.cc:460] Webserver started at http://127.0.206.126:37069/ using document root <none> and password file <none>
I20260619 10:17:22.935688 4341 fs_manager.cc:359] Metadata directory not provided
I20260619 10:17:22.935727 4341 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 10:17:22.935813 4341 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 10:17:22.936802 4341 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781864224577058-825-0/minicluster-data/master-0/data/instance:
uuid: "6c20cc2530884cd19277f4def2936e91"
format_stamp: "Formatted at 2026-06-19 10:17:22 on dist-test-slave-6pt2"
server_key: "6c07a1b01bda1f4aac3d3ae0375dfeed"
server_key_iv: "6b60e0acd46629142a9c0e3c0bdc1113"
server_key_version: "encryptionkey@0"
I20260619 10:17:22.937096 4341 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781864224577058-825-0/minicluster-data/master-0/wal/instance:
uuid: "6c20cc2530884cd19277f4def2936e91"
format_stamp: "Formatted at 2026-06-19 10:17:22 on dist-test-slave-6pt2"
server_key: "6c07a1b01bda1f4aac3d3ae0375dfeed"
server_key_iv: "6b60e0acd46629142a9c0e3c0bdc1113"
server_key_version: "encryptionkey@0"
I20260619 10:17:22.938313 4341 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 10:17:22.939034 4356 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 10:17:22.939180 4341 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260619 10:17:22.939251 4341 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781864224577058-825-0/minicluster-data/master-0/data,/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781864224577058-825-0/minicluster-data/master-0/wal
uuid: "6c20cc2530884cd19277f4def2936e91"
format_stamp: "Formatted at 2026-06-19 10:17:22 on dist-test-slave-6pt2"
server_key: "6c07a1b01bda1f4aac3d3ae0375dfeed"
server_key_iv: "6b60e0acd46629142a9c0e3c0bdc1113"
server_key_version: "encryptionkey@0"
I20260619 10:17:22.939344 4341 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781864224577058-825-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781864224577058-825-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781864224577058-825-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 10:17:22.952337 4341 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 10:17:22.954182 4341 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 10:17:22.954332 4341 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 10:17:22.958209 4341 rpc_server.cc:307] RPC server started. Bound to: 127.0.206.126:34083
I20260619 10:17:22.958257 4408 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.206.126:34083 every 8 connection(s)
I20260619 10:17:22.958681 4341 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781864224577058-825-0/minicluster-data/master-0/data/info.pb
I20260619 10:17:22.959584 4409 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 10:17:22.961798 4409 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6c20cc2530884cd19277f4def2936e91: Bootstrap starting.
I20260619 10:17:22.962378 4409 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6c20cc2530884cd19277f4def2936e91: Neither blocks nor log segments found. Creating new log.
I20260619 10:17:22.962738 4409 log.cc:824] T 00000000000000000000000000000000 P 6c20cc2530884cd19277f4def2936e91: Log is configured to *not* fsync() on all Append() calls
I20260619 10:17:22.963402 4409 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6c20cc2530884cd19277f4def2936e91: No bootstrap required, opened a new log
I20260619 10:17:22.964048 825 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTU9OP6/build/release/bin/kudu as pid 4341
I20260619 10:17:22.964136 825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781864224577058-825-0/minicluster-data/master-0/wal/instance
I20260619 10:17:22.964314 825 external_mini_cluster.cc:1428] Setting key 462d8b9a31f03560861710ca1d77d4c7
I20260619 10:17:22.965277 4409 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6c20cc2530884cd19277f4def2936e91 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c20cc2530884cd19277f4def2936e91" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 34083 } }
I20260619 10:17:22.965430 4409 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6c20cc2530884cd19277f4def2936e91 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 10:17:22.965492 4409 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6c20cc2530884cd19277f4def2936e91 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6c20cc2530884cd19277f4def2936e91, State: Initialized, Role: FOLLOWER
I20260619 10:17:22.965624 4409 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6c20cc2530884cd19277f4def2936e91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c20cc2530884cd19277f4def2936e91" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 34083 } }
I20260619 10:17:22.965709 4409 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6c20cc2530884cd19277f4def2936e91 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 10:17:22.965754 4409 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6c20cc2530884cd19277f4def2936e91 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 10:17:22.965843 4409 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6c20cc2530884cd19277f4def2936e91 [term 0 FOLLOWER]: Advancing to term 1
I20260619 10:17:22.966732 4409 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6c20cc2530884cd19277f4def2936e91 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c20cc2530884cd19277f4def2936e91" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 34083 } }
I20260619 10:17:22.966848 4409 leader_election.cc:302] T 00000000000000000000000000000000 P 6c20cc2530884cd19277f4def2936e91 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6c20cc2530884cd19277f4def2936e91; no voters:
I20260619 10:17:22.967012 4409 leader_election.cc:288] T 00000000000000000000000000000000 P 6c20cc2530884cd19277f4def2936e91 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 10:17:22.967038 4414 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6c20cc2530884cd19277f4def2936e91 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 10:17:22.967139 4414 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6c20cc2530884cd19277f4def2936e91 [term 1 LEADER]: Becoming Leader. State: Replica: 6c20cc2530884cd19277f4def2936e91, State: Running, Role: LEADER
I20260619 10:17:22.967257 4409 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6c20cc2530884cd19277f4def2936e91 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 10:17:22.967284 4414 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6c20cc2530884cd19277f4def2936e91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c20cc2530884cd19277f4def2936e91" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 34083 } }
I20260619 10:17:22.967605 4416 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6c20cc2530884cd19277f4def2936e91 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6c20cc2530884cd19277f4def2936e91" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c20cc2530884cd19277f4def2936e91" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 34083 } } }
I20260619 10:17:22.967705 4416 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6c20cc2530884cd19277f4def2936e91 [sys.catalog]: This master's current role is: LEADER
I20260619 10:17:22.967593 4414 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6c20cc2530884cd19277f4def2936e91 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6c20cc2530884cd19277f4def2936e91. Latest consensus state: current_term: 1 leader_uuid: "6c20cc2530884cd19277f4def2936e91" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c20cc2530884cd19277f4def2936e91" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 34083 } } }
I20260619 10:17:22.967896 4414 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6c20cc2530884cd19277f4def2936e91 [sys.catalog]: This master's current role is: LEADER
I20260619 10:17:22.968024 4421 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 10:17:22.968508 4421 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 10:17:22.970063 4421 catalog_manager.cc:1347] Generated new cluster ID: dff3bc0f590d4990a07e3325baec580f
I20260619 10:17:22.970115 4421 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Jun 19 10:17:22 dist-test-slave-6pt2 krb5kdc[4328](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864242, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.206.126@KRBTEST.COM
I20260619 10:17:22.974924 4413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:22.964830 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34066 (local address 127.0.206.126:34083)
0619 10:17:22.965029 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0619 10:17:22.965038 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 10:17:22.965069 (+ 31us) server_negotiation.cc:407] Connection header received
0619 10:17:22.965242 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:17:22.965247 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:17:22.965288 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:17:22.965416 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:17:22.966207 (+ 791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:22.967361 (+ 1154us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:22.968398 (+ 1037us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:22.969109 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:22.969234 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:22.971679 (+ 2445us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 10:17:22.971683 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 10:17:22.971683 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 10:17:22.971694 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0619 10:17:22.973358 (+ 1664us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:17:22.973737 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:17:22.973740 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:17:22.973740 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:17:22.973839 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:17:22.974047 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:17:22.974048 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:17:22.974048 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:17:22.974276 (+ 228us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 10:17:22.974440 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0619 10:17:22.974533 (+ 93us) server_negotiation.cc:299] Negotiation successful
0619 10:17:22.974689 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":147,"thread_start_us":90,"threads_started":1}
I20260619 10:17:22.983083 4421 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 10:17:22.983744 4421 catalog_manager.cc:1504] Loading token signing keys...
I20260619 10:17:22.994480 4421 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6c20cc2530884cd19277f4def2936e91: Generated new TSK 0
I20260619 10:17:22.994684 4421 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.0.206.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.206.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.206.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781864224577058-825-0/krb5kdc/kudu_127.0.206.65.keytab.
Entry for principal kudu/127.0.206.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781864224577058-825-0/krb5kdc/kudu_127.0.206.65.keytab.
I20260619 10:17:23.019580 825 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTU9OP6/build/release/bin/kudu
/tmp/dist-test-taskTU9OP6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781864224577058-825-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781864224577058-825-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781864224577058-825-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781864224577058-825-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.206.65:0
--local_ip_for_outbound_sockets=127.0.206.65
--webserver_interface=127.0.206.65
--webserver_port=0
--tserver_master_addrs=127.0.206.126:34083
--keytab_file=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781864224577058-825-0/krb5kdc/kudu_127.0.206.65.keytab
--principal=kudu/127.0.206.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.206.84:40891
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781864224577058-825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781864224577058-825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781864224577058-825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 10:17:23.100544 4435 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 10:17:23.100780 4435 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 10:17:23.100821 4435 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 10:17:23.101975 4435 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 10:17:23.102041 4435 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 10:17:23.102087 4435 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.206.65
I20260619 10:17:23.103647 4435 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.206.84:40891
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781864224577058-825-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781864224577058-825-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781864224577058-825-0/krb5kdc/kudu_127.0.206.65.keytab
--principal=kudu/127.0.206.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.206.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781864224577058-825-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.206.65
--webserver_port=0
--tserver_master_addrs=127.0.206.126:34083
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4435
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.206.65
--log_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781864224577058-825-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 09:32:44 UTC on ccycloud.quasar-gbtldp.root.comops.site
build id 4455440
I20260619 10:17:23.103915 4435 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 10:17:23.104140 4435 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 10:17:23.109020 4440 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 10:17:23.109018 4443 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 10:17:23.109030 4441 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 10:17:23.109257 4435 server_base.cc:1034] running on GCE node
I20260619 10:17:23.109440 4435 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 10:17:23.109660 4435 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 10:17:23.110816 4435 hybrid_clock.cc:648] HybridClock initialized: now 1781864243110799 us; error 37 us; skew 500 ppm
Jun 19 10:17:23 dist-test-slave-6pt2 krb5kdc[4328](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864243, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 10:17:23.113232 4435 init.cc:377] Logged in from keytab as kudu/127.0.206.65@KRBTEST.COM (short username kudu)
I20260619 10:17:23.113623 4435 webserver.cc:460] Webserver started at http://127.0.206.65:39743/ using document root <none> and password file <none>
I20260619 10:17:23.113854 4435 fs_manager.cc:359] Metadata directory not provided
I20260619 10:17:23.113919 4435 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 10:17:23.114064 4435 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 10:17:23.115131 4435 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781864224577058-825-0/minicluster-data/ts-0/data/instance:
uuid: "26821b7d11654b8aa3cc30b74ada2973"
format_stamp: "Formatted at 2026-06-19 10:17:23 on dist-test-slave-6pt2"
server_key: "309cef7fccf4210ec8df42190f002420"
server_key_iv: "4bbac4c90ec6cd80137613733c2197bf"
server_key_version: "encryptionkey@0"
I20260619 10:17:23.115482 4435 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781864224577058-825-0/minicluster-data/ts-0/wal/instance:
uuid: "26821b7d11654b8aa3cc30b74ada2973"
format_stamp: "Formatted at 2026-06-19 10:17:23 on dist-test-slave-6pt2"
server_key: "309cef7fccf4210ec8df42190f002420"
server_key_iv: "4bbac4c90ec6cd80137613733c2197bf"
server_key_version: "encryptionkey@0"
I20260619 10:17:23.116786 4435 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 10:17:23.117543 4450 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 10:17:23.117792 4435 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 10:17:23.117913 4435 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781864224577058-825-0/minicluster-data/ts-0/data,/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781864224577058-825-0/minicluster-data/ts-0/wal
uuid: "26821b7d11654b8aa3cc30b74ada2973"
format_stamp: "Formatted at 2026-06-19 10:17:23 on dist-test-slave-6pt2"
server_key: "309cef7fccf4210ec8df42190f002420"
server_key_iv: "4bbac4c90ec6cd80137613733c2197bf"
server_key_version: "encryptionkey@0"
I20260619 10:17:23.117985 4435 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781864224577058-825-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781864224577058-825-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781864224577058-825-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 10:17:23.129266 4435 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 10:17:23.131105 4435 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 10:17:23.131256 4435 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 10:17:23.131469 4435 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 10:17:23.131757 4435 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 10:17:23.131809 4435 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 10:17:23.131850 4435 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 10:17:23.131877 4435 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 10:17:23.137676 4435 rpc_server.cc:307] RPC server started. Bound to: 127.0.206.65:42399
I20260619 10:17:23.137722 4563 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.206.65:42399 every 8 connection(s)
I20260619 10:17:23.138015 4435 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781864224577058-825-0/minicluster-data/ts-0/data/info.pb
Jun 19 10:17:23 dist-test-slave-6pt2 krb5kdc[4328](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864243, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.65@KRBTEST.COM for kudu/127.0.206.126@KRBTEST.COM
I20260619 10:17:23.144769 825 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTU9OP6/build/release/bin/kudu as pid 4435
I20260619 10:17:23.144881 825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781864224577058-825-0/minicluster-data/ts-0/wal/instance
I20260619 10:17:23.145054 825 external_mini_cluster.cc:1428] Setting key 1ab6c555e6de0b24e2f56833252a0e0a
I20260619 10:17:23.147796 4413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:23.138757 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.65:35531 (local address 127.0.206.126:34083)
0619 10:17:23.138820 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0619 10:17:23.138822 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 10:17:23.139020 (+ 198us) server_negotiation.cc:407] Connection header received
0619 10:17:23.139992 (+ 972us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:17:23.139994 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:17:23.140010 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:17:23.140068 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:17:23.140725 (+ 657us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:23.141236 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:23.142326 (+ 1090us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:23.142540 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:23.142629 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:23.144598 (+ 1969us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 10:17:23.144600 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 10:17:23.144600 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 10:17:23.144612 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0619 10:17:23.146335 (+ 1723us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:17:23.146792 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:17:23.146794 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:17:23.146794 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:17:23.146858 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:17:23.147082 (+ 224us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:17:23.147083 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:17:23.147084 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:17:23.147229 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 10:17:23.147304 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0619 10:17:23.147652 (+ 348us) server_negotiation.cc:299] Negotiation successful
0619 10:17:23.147730 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260619 10:17:23.148064 4566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:23.138747 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:34083 (local address 127.0.206.65:35531)
0619 10:17:23.138890 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0619 10:17:23.138934 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0619 10:17:23.139879 (+ 945us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 10:17:23.140148 (+ 269us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 10:17:23.140152 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 10:17:23.140422 (+ 270us) client_negotiation.cc:189] Negotiated authn=SASL
0619 10:17:23.140614 (+ 192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:23.140618 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:23.141330 (+ 712us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:23.141331 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:23.142228 (+ 897us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:23.142231 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:23.142633 (+ 402us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:23.142634 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:23.142756 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:23.143206 (+ 450us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 10:17:23.143214 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 10:17:23.144494 (+ 1280us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 10:17:23.146414 (+ 1920us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 10:17:23.146419 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 10:17:23.146421 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 10:17:23.146724 (+ 303us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 10:17:23.146925 (+ 201us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 10:17:23.146926 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 10:17:23.146926 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 10:17:23.147020 (+ 94us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 10:17:23.147304 (+ 284us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 10:17:23.147310 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 10:17:23.147482 (+ 172us) client_negotiation.cc:769] Sending connection context
0619 10:17:23.147605 (+ 123us) client_negotiation.cc:240] Negotiation successful
0619 10:17:23.147805 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":70,"threads_started":1}
I20260619 10:17:23.148661 4564 heartbeater.cc:347] Connected to a master server at 127.0.206.126:34083
I20260619 10:17:23.148744 4564 heartbeater.cc:464] Registering TS with master...
I20260619 10:17:23.148928 4564 heartbeater.cc:511] Master 127.0.206.126:34083 requested a full tablet report, sending...
I20260619 10:17:23.149449 4373 ts_manager.cc:194] Registered new tserver with Master: 26821b7d11654b8aa3cc30b74ada2973 (127.0.206.65:42399)
I20260619 10:17:23.150084 4373 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.206.65@KRBTEST.COM'} at 127.0.206.65:35531
WARNING: no policy specified for kudu/127.0.206.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.206.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.206.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781864224577058-825-0/krb5kdc/kudu_127.0.206.66.keytab.
Entry for principal kudu/127.0.206.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781864224577058-825-0/krb5kdc/kudu_127.0.206.66.keytab.
I20260619 10:17:23.169651 825 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTU9OP6/build/release/bin/kudu
/tmp/dist-test-taskTU9OP6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781864224577058-825-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781864224577058-825-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781864224577058-825-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781864224577058-825-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.206.66:0
--local_ip_for_outbound_sockets=127.0.206.66
--webserver_interface=127.0.206.66
--webserver_port=0
--tserver_master_addrs=127.0.206.126:34083
--keytab_file=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781864224577058-825-0/krb5kdc/kudu_127.0.206.66.keytab
--principal=kudu/127.0.206.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.206.84:40891
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781864224577058-825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781864224577058-825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781864224577058-825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 10:17:23.250708 4569 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 10:17:23.250972 4569 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 10:17:23.250998 4569 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 10:17:23.252146 4569 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 10:17:23.252180 4569 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 10:17:23.252205 4569 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.206.66
I20260619 10:17:23.253688 4569 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.206.84:40891
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781864224577058-825-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781864224577058-825-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781864224577058-825-0/krb5kdc/kudu_127.0.206.66.keytab
--principal=kudu/127.0.206.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.206.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781864224577058-825-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.206.66
--webserver_port=0
--tserver_master_addrs=127.0.206.126:34083
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4569
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.206.66
--log_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781864224577058-825-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 09:32:44 UTC on ccycloud.quasar-gbtldp.root.comops.site
build id 4455440
I20260619 10:17:23.253868 4569 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 10:17:23.254031 4569 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 10:17:23.258891 4574 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 10:17:23.258941 4575 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 10:17:23.258960 4577 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 10:17:23.259121 4569 server_base.cc:1034] running on GCE node
I20260619 10:17:23.259321 4569 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 10:17:23.259548 4569 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 10:17:23.260690 4569 hybrid_clock.cc:648] HybridClock initialized: now 1781864243260675 us; error 27 us; skew 500 ppm
Jun 19 10:17:23 dist-test-slave-6pt2 krb5kdc[4328](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864243, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 10:17:23.263342 4569 init.cc:377] Logged in from keytab as kudu/127.0.206.66@KRBTEST.COM (short username kudu)
I20260619 10:17:23.263757 4569 webserver.cc:460] Webserver started at http://127.0.206.66:41167/ using document root <none> and password file <none>
I20260619 10:17:23.263980 4569 fs_manager.cc:359] Metadata directory not provided
I20260619 10:17:23.264047 4569 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 10:17:23.264205 4569 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 10:17:23.265209 4569 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781864224577058-825-0/minicluster-data/ts-1/data/instance:
uuid: "c52bc3ea07ec4970a306914047c1af60"
format_stamp: "Formatted at 2026-06-19 10:17:23 on dist-test-slave-6pt2"
server_key: "415660a5b1c5e662e6bec0ecb54f756a"
server_key_iv: "326b8c7ebeca3ac46f0f0b01d9f1c430"
server_key_version: "encryptionkey@0"
I20260619 10:17:23.265570 4569 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781864224577058-825-0/minicluster-data/ts-1/wal/instance:
uuid: "c52bc3ea07ec4970a306914047c1af60"
format_stamp: "Formatted at 2026-06-19 10:17:23 on dist-test-slave-6pt2"
server_key: "415660a5b1c5e662e6bec0ecb54f756a"
server_key_iv: "326b8c7ebeca3ac46f0f0b01d9f1c430"
server_key_version: "encryptionkey@0"
I20260619 10:17:23.266906 4569 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260619 10:17:23.267778 4584 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 10:17:23.267961 4569 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260619 10:17:23.268059 4569 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781864224577058-825-0/minicluster-data/ts-1/data,/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781864224577058-825-0/minicluster-data/ts-1/wal
uuid: "c52bc3ea07ec4970a306914047c1af60"
format_stamp: "Formatted at 2026-06-19 10:17:23 on dist-test-slave-6pt2"
server_key: "415660a5b1c5e662e6bec0ecb54f756a"
server_key_iv: "326b8c7ebeca3ac46f0f0b01d9f1c430"
server_key_version: "encryptionkey@0"
I20260619 10:17:23.268152 4569 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781864224577058-825-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781864224577058-825-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781864224577058-825-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 10:17:23.280326 4569 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 10:17:23.281991 4569 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 10:17:23.282143 4569 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 10:17:23.282342 4569 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 10:17:23.282668 4569 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 10:17:23.282738 4569 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 10:17:23.282783 4569 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 10:17:23.282809 4569 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 10:17:23.288530 4569 rpc_server.cc:307] RPC server started. Bound to: 127.0.206.66:39407
I20260619 10:17:23.288563 4697 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.206.66:39407 every 8 connection(s)
I20260619 10:17:23.288866 4569 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781864224577058-825-0/minicluster-data/ts-1/data/info.pb
I20260619 10:17:23.294340 825 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTU9OP6/build/release/bin/kudu as pid 4569
I20260619 10:17:23.294418 825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781864224577058-825-0/minicluster-data/ts-1/wal/instance
I20260619 10:17:23.294651 825 external_mini_cluster.cc:1428] Setting key 6b7c4a8f9befcc48cc94eac69f655f40
Jun 19 10:17:23 dist-test-slave-6pt2 krb5kdc[4328](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864243, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.66@KRBTEST.COM for kudu/127.0.206.126@KRBTEST.COM
I20260619 10:17:23.298261 4700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:23.289535 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:34083 (local address 127.0.206.66:43477)
0619 10:17:23.289690 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0619 10:17:23.289729 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0619 10:17:23.290736 (+ 1007us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 10:17:23.290947 (+ 211us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 10:17:23.290950 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 10:17:23.291186 (+ 236us) client_negotiation.cc:189] Negotiated authn=SASL
0619 10:17:23.291370 (+ 184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:23.291374 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:23.292001 (+ 627us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:23.292002 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:23.292789 (+ 787us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:23.292790 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:23.293110 (+ 320us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:23.293111 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:23.293203 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:23.293626 (+ 423us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 10:17:23.293634 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 10:17:23.295079 (+ 1445us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 10:17:23.297019 (+ 1940us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 10:17:23.297025 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 10:17:23.297026 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 10:17:23.297258 (+ 232us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 10:17:23.297464 (+ 206us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 10:17:23.297465 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 10:17:23.297465 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 10:17:23.297554 (+ 89us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 10:17:23.297805 (+ 251us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 10:17:23.297808 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 10:17:23.297910 (+ 102us) client_negotiation.cc:769] Sending connection context
0619 10:17:23.297987 (+ 77us) client_negotiation.cc:240] Negotiation successful
0619 10:17:23.298093 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":88,"threads_started":1}
I20260619 10:17:23.298280 4413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:23.289580 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.66:43477 (local address 127.0.206.126:34083)
0619 10:17:23.289625 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0619 10:17:23.289626 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 10:17:23.289824 (+ 198us) server_negotiation.cc:407] Connection header received
0619 10:17:23.290823 (+ 999us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:17:23.290824 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:17:23.290832 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:17:23.290884 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:17:23.291439 (+ 555us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:23.291926 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:23.292863 (+ 937us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:23.293042 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:23.293091 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:23.295178 (+ 2087us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 10:17:23.295179 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 10:17:23.295180 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 10:17:23.295186 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 10:17:23.296914 (+ 1728us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:17:23.297331 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:17:23.297333 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:17:23.297333 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:17:23.297397 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:17:23.297602 (+ 205us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:17:23.297603 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:17:23.297604 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:17:23.297743 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 10:17:23.297811 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0619 10:17:23.298029 (+ 218us) server_negotiation.cc:299] Negotiation successful
0619 10:17:23.298125 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260619 10:17:23.299085 4698 heartbeater.cc:347] Connected to a master server at 127.0.206.126:34083
I20260619 10:17:23.299198 4698 heartbeater.cc:464] Registering TS with master...
I20260619 10:17:23.299366 4698 heartbeater.cc:511] Master 127.0.206.126:34083 requested a full tablet report, sending...
I20260619 10:17:23.299754 4373 ts_manager.cc:194] Registered new tserver with Master: c52bc3ea07ec4970a306914047c1af60 (127.0.206.66:39407)
I20260619 10:17:23.300133 4373 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.206.66@KRBTEST.COM'} at 127.0.206.66:43477
WARNING: no policy specified for kudu/127.0.206.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.206.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.206.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781864224577058-825-0/krb5kdc/kudu_127.0.206.67.keytab.
Entry for principal kudu/127.0.206.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781864224577058-825-0/krb5kdc/kudu_127.0.206.67.keytab.
I20260619 10:17:23.318722 825 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTU9OP6/build/release/bin/kudu
/tmp/dist-test-taskTU9OP6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781864224577058-825-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781864224577058-825-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781864224577058-825-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781864224577058-825-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.206.67:0
--local_ip_for_outbound_sockets=127.0.206.67
--webserver_interface=127.0.206.67
--webserver_port=0
--tserver_master_addrs=127.0.206.126:34083
--keytab_file=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781864224577058-825-0/krb5kdc/kudu_127.0.206.67.keytab
--principal=kudu/127.0.206.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.206.84:40891
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781864224577058-825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781864224577058-825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781864224577058-825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 10:17:23.399044 4703 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 10:17:23.399236 4703 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 10:17:23.399259 4703 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 10:17:23.400421 4703 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 10:17:23.400463 4703 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 10:17:23.400518 4703 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.206.67
I20260619 10:17:23.401971 4703 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.206.84:40891
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781864224577058-825-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781864224577058-825-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781864224577058-825-0/krb5kdc/kudu_127.0.206.67.keytab
--principal=kudu/127.0.206.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.206.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781864224577058-825-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.206.67
--webserver_port=0
--tserver_master_addrs=127.0.206.126:34083
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4703
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.206.67
--log_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781864224577058-825-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 09:32:44 UTC on ccycloud.quasar-gbtldp.root.comops.site
build id 4455440
I20260619 10:17:23.402209 4703 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 10:17:23.402428 4703 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 10:17:23.407491 4711 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 10:17:23.407475 4708 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 10:17:23.407521 4703 server_base.cc:1034] running on GCE node
W20260619 10:17:23.407492 4709 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 10:17:23.407915 4703 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 10:17:23.408147 4703 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 10:17:23.409305 4703 hybrid_clock.cc:648] HybridClock initialized: now 1781864243409289 us; error 30 us; skew 500 ppm
Jun 19 10:17:23 dist-test-slave-6pt2 krb5kdc[4328](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864243, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 10:17:23.411760 4703 init.cc:377] Logged in from keytab as kudu/127.0.206.67@KRBTEST.COM (short username kudu)
I20260619 10:17:23.412189 4703 webserver.cc:460] Webserver started at http://127.0.206.67:34533/ using document root <none> and password file <none>
I20260619 10:17:23.412365 4703 fs_manager.cc:359] Metadata directory not provided
I20260619 10:17:23.412405 4703 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 10:17:23.412487 4703 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 10:17:23.413455 4703 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781864224577058-825-0/minicluster-data/ts-2/data/instance:
uuid: "a4eceba3141f4bf489d4cac2fa909e9f"
format_stamp: "Formatted at 2026-06-19 10:17:23 on dist-test-slave-6pt2"
server_key: "7682265e7ad19e49dfcccdf7ecc87366"
server_key_iv: "6e94555be38941f16f1ac073d53c6577"
server_key_version: "encryptionkey@0"
I20260619 10:17:23.413735 4703 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781864224577058-825-0/minicluster-data/ts-2/wal/instance:
uuid: "a4eceba3141f4bf489d4cac2fa909e9f"
format_stamp: "Formatted at 2026-06-19 10:17:23 on dist-test-slave-6pt2"
server_key: "7682265e7ad19e49dfcccdf7ecc87366"
server_key_iv: "6e94555be38941f16f1ac073d53c6577"
server_key_version: "encryptionkey@0"
I20260619 10:17:23.414999 4703 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 10:17:23.415657 4718 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 10:17:23.415838 4703 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 10:17:23.415923 4703 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781864224577058-825-0/minicluster-data/ts-2/data,/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781864224577058-825-0/minicluster-data/ts-2/wal
uuid: "a4eceba3141f4bf489d4cac2fa909e9f"
format_stamp: "Formatted at 2026-06-19 10:17:23 on dist-test-slave-6pt2"
server_key: "7682265e7ad19e49dfcccdf7ecc87366"
server_key_iv: "6e94555be38941f16f1ac073d53c6577"
server_key_version: "encryptionkey@0"
I20260619 10:17:23.415994 4703 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781864224577058-825-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781864224577058-825-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781864224577058-825-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 10:17:23.427098 4703 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 10:17:23.428732 4703 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 10:17:23.428884 4703 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 10:17:23.429096 4703 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 10:17:23.429376 4703 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 10:17:23.429409 4703 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 10:17:23.429430 4703 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 10:17:23.429477 4703 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 10:17:23.435148 4703 rpc_server.cc:307] RPC server started. Bound to: 127.0.206.67:38017
I20260619 10:17:23.435205 4831 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.206.67:38017 every 8 connection(s)
I20260619 10:17:23.435513 4703 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781864224577058-825-0/minicluster-data/ts-2/data/info.pb
Jun 19 10:17:23 dist-test-slave-6pt2 krb5kdc[4328](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864243, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.67@KRBTEST.COM for kudu/127.0.206.126@KRBTEST.COM
I20260619 10:17:23.443327 825 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTU9OP6/build/release/bin/kudu as pid 4703
I20260619 10:17:23.443439 825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781864224577058-825-0/minicluster-data/ts-2/wal/instance
I20260619 10:17:23.443624 825 external_mini_cluster.cc:1428] Setting key 5ca80c7450fbb463f5e6e7ddc6e2594c
I20260619 10:17:23.444968 4413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:23.436234 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.67:42499 (local address 127.0.206.126:34083)
0619 10:17:23.436316 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0619 10:17:23.436318 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 10:17:23.436470 (+ 152us) server_negotiation.cc:407] Connection header received
0619 10:17:23.437404 (+ 934us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:17:23.437405 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:17:23.437417 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:17:23.437468 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:17:23.438033 (+ 565us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:23.438524 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:23.439545 (+ 1021us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:23.439759 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:23.439855 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:23.442168 (+ 2313us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 10:17:23.442170 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 10:17:23.442170 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 10:17:23.442176 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 10:17:23.443560 (+ 1384us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:17:23.443994 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:17:23.443995 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:17:23.443995 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:17:23.444054 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:17:23.444315 (+ 261us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:17:23.444316 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:17:23.444317 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:17:23.444455 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 10:17:23.444529 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0619 10:17:23.444799 (+ 270us) server_negotiation.cc:299] Negotiation successful
0619 10:17:23.444879 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260619 10:17:23.445156 4834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:23.436165 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:34083 (local address 127.0.206.67:42499)
0619 10:17:23.436354 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0619 10:17:23.436392 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0619 10:17:23.437302 (+ 910us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 10:17:23.437531 (+ 229us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 10:17:23.437535 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 10:17:23.437770 (+ 235us) client_negotiation.cc:189] Negotiated authn=SASL
0619 10:17:23.437943 (+ 173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:23.437947 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:23.438640 (+ 693us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:23.438641 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:23.439455 (+ 814us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:23.439457 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:23.439861 (+ 404us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:23.439862 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:23.439979 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:23.440464 (+ 485us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 10:17:23.440472 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 10:17:23.442083 (+ 1611us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 10:17:23.443633 (+ 1550us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 10:17:23.443639 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 10:17:23.443641 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 10:17:23.443900 (+ 259us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 10:17:23.444131 (+ 231us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 10:17:23.444132 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 10:17:23.444132 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 10:17:23.444271 (+ 139us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 10:17:23.444532 (+ 261us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 10:17:23.444539 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 10:17:23.444669 (+ 130us) client_negotiation.cc:769] Sending connection context
0619 10:17:23.444776 (+ 107us) client_negotiation.cc:240] Negotiation successful
0619 10:17:23.444927 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":84,"threads_started":1}
I20260619 10:17:23.445596 4832 heartbeater.cc:347] Connected to a master server at 127.0.206.126:34083
I20260619 10:17:23.445670 4832 heartbeater.cc:464] Registering TS with master...
I20260619 10:17:23.445853 4832 heartbeater.cc:511] Master 127.0.206.126:34083 requested a full tablet report, sending...
I20260619 10:17:23.446204 4373 ts_manager.cc:194] Registered new tserver with Master: a4eceba3141f4bf489d4cac2fa909e9f (127.0.206.67:38017)
I20260619 10:17:23.446592 4373 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.206.67@KRBTEST.COM'} at 127.0.206.67:42499
I20260619 10:17:23.455472 825 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Jun 19 10:17:23 dist-test-slave-6pt2 krb5kdc[4328](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864243, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Jun 19 10:17:23 dist-test-slave-6pt2 krb5kdc[4328](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864243, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.206.65@KRBTEST.COM
I20260619 10:17:23.477789 4839 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:23.469905 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52226 (local address 127.0.206.65:42399)
0619 10:17:23.470062 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0619 10:17:23.470068 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 10:17:23.470084 (+ 16us) server_negotiation.cc:407] Connection header received
0619 10:17:23.470106 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:17:23.470108 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:17:23.470137 (+ 29us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:17:23.470217 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:17:23.471014 (+ 797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:23.471826 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:23.472640 (+ 814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:23.472947 (+ 307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:23.473048 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:23.475395 (+ 2347us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 10:17:23.475401 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 10:17:23.475402 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 10:17:23.475408 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 10:17:23.476777 (+ 1369us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:17:23.477108 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:17:23.477111 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:17:23.477111 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:17:23.477169 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:17:23.477355 (+ 186us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:17:23.477356 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:17:23.477356 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:17:23.477475 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 10:17:23.477574 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0619 10:17:23.477638 (+ 64us) server_negotiation.cc:299] Negotiation successful
0619 10:17:23.477703 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":122,"thread_start_us":76,"threads_started":1}
Jun 19 10:17:23 dist-test-slave-6pt2 krb5kdc[4328](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864243, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.206.126@KRBTEST.COM
I20260619 10:17:23.485118 4413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:23.479026 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34068 (local address 127.0.206.126:34083)
0619 10:17:23.479093 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0619 10:17:23.479095 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 10:17:23.479106 (+ 11us) server_negotiation.cc:407] Connection header received
0619 10:17:23.479131 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:17:23.479132 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:17:23.479143 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:17:23.479221 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:17:23.479689 (+ 468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:23.480161 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:23.480830 (+ 669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:23.481024 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:23.481100 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:23.483076 (+ 1976us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 10:17:23.483077 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 10:17:23.483078 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 10:17:23.483083 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0619 10:17:23.484227 (+ 1144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:17:23.484507 (+ 280us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:17:23.484508 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:17:23.484508 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:17:23.484554 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:17:23.484690 (+ 136us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:17:23.484691 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:17:23.484691 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:17:23.484807 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 10:17:23.484883 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0619 10:17:23.484960 (+ 77us) server_negotiation.cc:299] Negotiation successful
0619 10:17:23.485017 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
W20260619 10:17:23.485381 4373 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34068
I20260619 10:17:23.485769 825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTU9OP6/build/release/bin/kudu with pid 4435
I20260619 10:17:23.490484 825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTU9OP6/build/release/bin/kudu with pid 4569
I20260619 10:17:23.494571 825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTU9OP6/build/release/bin/kudu with pid 4703
I20260619 10:17:23.498695 825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTU9OP6/build/release/bin/kudu with pid 4341
2026-06-19T10:17:23Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (1853 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781864224577058-825-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 10:17:23 dist-test-slave-6pt2 krb5kdc[4844](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 10:17:23 dist-test-slave-6pt2 krb5kdc[4844](info): set up 2 sockets
Jun 19 10:17:23 dist-test-slave-6pt2 krb5kdc[4844](info): commencing operation
krb5kdc: starting...
W20260619 10:17:24.538419 825 mini_kdc.cc:121] Time spent starting KDC: real 1.020s user 0.001s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781864224577058-825-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781864224577058-825-0/krb5kdc/test-user.keytab.
Jun 19 10:17:24 dist-test-slave-6pt2 krb5kdc[4844](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864244, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T10:17:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T10:17:24Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.206.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.206.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.206.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781864224577058-825-0/krb5kdc/kudu_127.0.206.126.keytab.
Entry for principal kudu/127.0.206.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781864224577058-825-0/krb5kdc/kudu_127.0.206.126.keytab.
I20260619 10:17:24.647423 825 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTU9OP6/build/release/bin/kudu
/tmp/dist-test-taskTU9OP6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781864224577058-825-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781864224577058-825-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781864224577058-825-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781864224577058-825-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.206.126:43221
--webserver_interface=127.0.206.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781864224577058-825-0/krb5kdc/kudu_127.0.206.126.keytab
--principal=kudu/127.0.206.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.206.84:35871
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.206.126:43221
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781864224577058-825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781864224577058-825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781864224577058-825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 10:17:24.725734 4857 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 10:17:24.725920 4857 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 10:17:24.725942 4857 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 10:17:24.727139 4857 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 10:17:24.727178 4857 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 10:17:24.727193 4857 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 10:17:24.727226 4857 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 10:17:24.727252 4857 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 10:17:24.728865 4857 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.206.84:35871
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781864224577058-825-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781864224577058-825-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.206.126:43221
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781864224577058-825-0/krb5kdc/kudu_127.0.206.126.keytab
--principal=kudu/127.0.206.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.206.126:43221
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781864224577058-825-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.206.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4857
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781864224577058-825-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 09:32:44 UTC on ccycloud.quasar-gbtldp.root.comops.site
build id 4455440
I20260619 10:17:24.729101 4857 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 10:17:24.729308 4857 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 10:17:24.734041 4865 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 10:17:24.734066 4862 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 10:17:24.734081 4857 server_base.cc:1034] running on GCE node
W20260619 10:17:24.734066 4863 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 10:17:24.734480 4857 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 10:17:24.734781 4857 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 10:17:24.735953 4857 hybrid_clock.cc:648] HybridClock initialized: now 1781864244735938 us; error 28 us; skew 500 ppm
Jun 19 10:17:24 dist-test-slave-6pt2 krb5kdc[4844](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864244, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 10:17:24.738377 4857 init.cc:377] Logged in from keytab as kudu/127.0.206.126@KRBTEST.COM (short username kudu)
I20260619 10:17:24.738750 4857 fs_manager.cc:359] Metadata directory not provided
I20260619 10:17:24.738797 4857 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 10:17:24.738940 4857 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 10:17:24.739951 4857 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781864224577058-825-0/minicluster-data/master-0/data/instance:
uuid: "221a9ce12eb94059a9fcb57a2fd812fc"
format_stamp: "Formatted at 2026-06-19 10:17:24 on dist-test-slave-6pt2"
server_key: "613de1f477f8a9e7095aaf746acbfebe"
server_key_iv: "8c5c6970ed5ba2364a7dfa72f4e30f1b"
server_key_version: "encryptionkey@0"
I20260619 10:17:24.740303 4857 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781864224577058-825-0/minicluster-data/master-0/wal/instance:
uuid: "221a9ce12eb94059a9fcb57a2fd812fc"
format_stamp: "Formatted at 2026-06-19 10:17:24 on dist-test-slave-6pt2"
server_key: "613de1f477f8a9e7095aaf746acbfebe"
server_key_iv: "8c5c6970ed5ba2364a7dfa72f4e30f1b"
server_key_version: "encryptionkey@0"
I20260619 10:17:24.741557 4857 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 10:17:24.742220 4871 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 10:17:24.742388 4857 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 10:17:24.742470 4857 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781864224577058-825-0/minicluster-data/master-0/data,/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781864224577058-825-0/minicluster-data/master-0/wal
uuid: "221a9ce12eb94059a9fcb57a2fd812fc"
format_stamp: "Formatted at 2026-06-19 10:17:24 on dist-test-slave-6pt2"
server_key: "613de1f477f8a9e7095aaf746acbfebe"
server_key_iv: "8c5c6970ed5ba2364a7dfa72f4e30f1b"
server_key_version: "encryptionkey@0"
I20260619 10:17:24.742523 4857 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781864224577058-825-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781864224577058-825-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781864224577058-825-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 10:17:24.754190 4857 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 10:17:24.756043 4857 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 10:17:24.756153 4857 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 10:17:24.759832 4857 rpc_server.cc:307] RPC server started. Bound to: 127.0.206.126:43221
I20260619 10:17:24.759855 4923 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.206.126:43221 every 8 connection(s)
I20260619 10:17:24.760141 4857 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781864224577058-825-0/minicluster-data/master-0/data/info.pb
I20260619 10:17:24.761070 4924 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 10:17:24.761659 825 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTU9OP6/build/release/bin/kudu as pid 4857
I20260619 10:17:24.761783 825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781864224577058-825-0/minicluster-data/master-0/wal/instance
I20260619 10:17:24.761955 825 external_mini_cluster.cc:1428] Setting key 4b17cbde5dd283cd2370855e40e1d494
I20260619 10:17:24.764215 4924 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 221a9ce12eb94059a9fcb57a2fd812fc: Bootstrap starting.
I20260619 10:17:24.764813 4924 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 221a9ce12eb94059a9fcb57a2fd812fc: Neither blocks nor log segments found. Creating new log.
I20260619 10:17:24.765091 4924 log.cc:824] T 00000000000000000000000000000000 P 221a9ce12eb94059a9fcb57a2fd812fc: Log is configured to *not* fsync() on all Append() calls
I20260619 10:17:24.765749 4924 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 221a9ce12eb94059a9fcb57a2fd812fc: No bootstrap required, opened a new log
I20260619 10:17:24.767634 4924 raft_consensus.cc:348] T 00000000000000000000000000000000 P 221a9ce12eb94059a9fcb57a2fd812fc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "221a9ce12eb94059a9fcb57a2fd812fc" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 43221 } }
I20260619 10:17:24.767835 4924 raft_consensus.cc:374] T 00000000000000000000000000000000 P 221a9ce12eb94059a9fcb57a2fd812fc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 10:17:24.767875 4924 raft_consensus.cc:729] T 00000000000000000000000000000000 P 221a9ce12eb94059a9fcb57a2fd812fc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 221a9ce12eb94059a9fcb57a2fd812fc, State: Initialized, Role: FOLLOWER
I20260619 10:17:24.767999 4924 consensus_queue.cc:260] T 00000000000000000000000000000000 P 221a9ce12eb94059a9fcb57a2fd812fc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "221a9ce12eb94059a9fcb57a2fd812fc" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 43221 } }
I20260619 10:17:24.768085 4924 raft_consensus.cc:388] T 00000000000000000000000000000000 P 221a9ce12eb94059a9fcb57a2fd812fc [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 10:17:24.768122 4924 raft_consensus.cc:482] T 00000000000000000000000000000000 P 221a9ce12eb94059a9fcb57a2fd812fc [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 10:17:24.768172 4924 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 221a9ce12eb94059a9fcb57a2fd812fc [term 0 FOLLOWER]: Advancing to term 1
Jun 19 10:17:24 dist-test-slave-6pt2 krb5kdc[4844](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864244, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.206.126@KRBTEST.COM
I20260619 10:17:24.768889 4924 raft_consensus.cc:504] T 00000000000000000000000000000000 P 221a9ce12eb94059a9fcb57a2fd812fc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "221a9ce12eb94059a9fcb57a2fd812fc" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 43221 } }
I20260619 10:17:24.768977 4924 leader_election.cc:302] T 00000000000000000000000000000000 P 221a9ce12eb94059a9fcb57a2fd812fc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 221a9ce12eb94059a9fcb57a2fd812fc; no voters:
I20260619 10:17:24.769148 4924 leader_election.cc:288] T 00000000000000000000000000000000 P 221a9ce12eb94059a9fcb57a2fd812fc [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 10:17:24.769236 4929 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 221a9ce12eb94059a9fcb57a2fd812fc [term 1 FOLLOWER]: Leader election won for term 1
I20260619 10:17:24.769382 4929 raft_consensus.cc:686] T 00000000000000000000000000000000 P 221a9ce12eb94059a9fcb57a2fd812fc [term 1 LEADER]: Becoming Leader. State: Replica: 221a9ce12eb94059a9fcb57a2fd812fc, State: Running, Role: LEADER
I20260619 10:17:24.769402 4924 sys_catalog.cc:564] T 00000000000000000000000000000000 P 221a9ce12eb94059a9fcb57a2fd812fc [sys.catalog]: configured and running, proceeding with master startup.
I20260619 10:17:24.769479 4929 consensus_queue.cc:237] T 00000000000000000000000000000000 P 221a9ce12eb94059a9fcb57a2fd812fc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "221a9ce12eb94059a9fcb57a2fd812fc" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 43221 } }
I20260619 10:17:24.769819 4930 sys_catalog.cc:455] T 00000000000000000000000000000000 P 221a9ce12eb94059a9fcb57a2fd812fc [sys.catalog]: SysCatalogTable state changed. Reason: New leader 221a9ce12eb94059a9fcb57a2fd812fc. Latest consensus state: current_term: 1 leader_uuid: "221a9ce12eb94059a9fcb57a2fd812fc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "221a9ce12eb94059a9fcb57a2fd812fc" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 43221 } } }
I20260619 10:17:24.769897 4930 sys_catalog.cc:458] T 00000000000000000000000000000000 P 221a9ce12eb94059a9fcb57a2fd812fc [sys.catalog]: This master's current role is: LEADER
I20260619 10:17:24.770131 4937 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 10:17:24.770591 4937 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 10:17:24.769862 4931 sys_catalog.cc:455] T 00000000000000000000000000000000 P 221a9ce12eb94059a9fcb57a2fd812fc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "221a9ce12eb94059a9fcb57a2fd812fc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "221a9ce12eb94059a9fcb57a2fd812fc" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 43221 } } }
I20260619 10:17:24.771121 4931 sys_catalog.cc:458] T 00000000000000000000000000000000 P 221a9ce12eb94059a9fcb57a2fd812fc [sys.catalog]: This master's current role is: LEADER
I20260619 10:17:24.772634 4937 catalog_manager.cc:1347] Generated new cluster ID: 278f368cc765417b9df3f1895cf102c0
I20260619 10:17:24.772676 4937 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 10:17:24.772862 4927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:24.762467 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34698 (local address 127.0.206.126:43221)
0619 10:17:24.762651 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0619 10:17:24.762658 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 10:17:24.762683 (+ 25us) server_negotiation.cc:407] Connection header received
0619 10:17:24.763081 (+ 398us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:17:24.763087 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:17:24.763129 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:17:24.763241 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:17:24.764004 (+ 763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:24.764930 (+ 926us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:24.765751 (+ 821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:24.766232 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:24.766346 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:24.768885 (+ 2539us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 10:17:24.768889 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 10:17:24.768889 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 10:17:24.768899 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0619 10:17:24.771040 (+ 2141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:17:24.771450 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:17:24.771453 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:17:24.771454 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:17:24.771554 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:17:24.771814 (+ 260us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:17:24.771815 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:17:24.771816 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:17:24.772068 (+ 252us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 10:17:24.772227 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0619 10:17:24.772454 (+ 227us) server_negotiation.cc:299] Negotiation successful
0619 10:17:24.772602 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":100,"thread_start_us":73,"threads_started":1}
I20260619 10:17:24.785421 4937 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 10:17:24.785985 4937 catalog_manager.cc:1504] Loading token signing keys...
I20260619 10:17:24.790444 4937 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 221a9ce12eb94059a9fcb57a2fd812fc: Generated new TSK 0
I20260619 10:17:24.790683 4937 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.0.206.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.206.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.206.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781864224577058-825-0/krb5kdc/kudu_127.0.206.65.keytab.
Entry for principal kudu/127.0.206.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781864224577058-825-0/krb5kdc/kudu_127.0.206.65.keytab.
I20260619 10:17:24.816692 825 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTU9OP6/build/release/bin/kudu
/tmp/dist-test-taskTU9OP6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781864224577058-825-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781864224577058-825-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781864224577058-825-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781864224577058-825-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.206.65:0
--local_ip_for_outbound_sockets=127.0.206.65
--webserver_interface=127.0.206.65
--webserver_port=0
--tserver_master_addrs=127.0.206.126:43221
--keytab_file=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781864224577058-825-0/krb5kdc/kudu_127.0.206.65.keytab
--principal=kudu/127.0.206.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.206.84:35871
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781864224577058-825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781864224577058-825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781864224577058-825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 10:17:24.896720 4950 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 10:17:24.896909 4950 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 10:17:24.896931 4950 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 10:17:24.898087 4950 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 10:17:24.898128 4950 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 10:17:24.898154 4950 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.206.65
I20260619 10:17:24.899619 4950 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.206.84:35871
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781864224577058-825-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781864224577058-825-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781864224577058-825-0/krb5kdc/kudu_127.0.206.65.keytab
--principal=kudu/127.0.206.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.206.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781864224577058-825-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.206.65
--webserver_port=0
--tserver_master_addrs=127.0.206.126:43221
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4950
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.206.65
--log_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781864224577058-825-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 09:32:44 UTC on ccycloud.quasar-gbtldp.root.comops.site
build id 4455440
I20260619 10:17:24.899871 4950 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 10:17:24.900100 4950 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 10:17:24.904942 4956 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 10:17:24.904956 4955 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 10:17:24.904955 4958 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 10:17:24.905092 4950 server_base.cc:1034] running on GCE node
I20260619 10:17:24.905371 4950 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 10:17:24.905578 4950 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 10:17:24.906742 4950 hybrid_clock.cc:648] HybridClock initialized: now 1781864244906733 us; error 38 us; skew 500 ppm
Jun 19 10:17:24 dist-test-slave-6pt2 krb5kdc[4844](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864244, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 10:17:24.909086 4950 init.cc:377] Logged in from keytab as kudu/127.0.206.65@KRBTEST.COM (short username kudu)
I20260619 10:17:24.909415 4950 fs_manager.cc:359] Metadata directory not provided
I20260619 10:17:24.909461 4950 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 10:17:24.909607 4950 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 10:17:24.910737 4950 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781864224577058-825-0/minicluster-data/ts-0/data/instance:
uuid: "4bcd6c8e17114dd8b3b4ea1f95dd5590"
format_stamp: "Formatted at 2026-06-19 10:17:24 on dist-test-slave-6pt2"
server_key: "4dcdb65acbdbba7ed853943fff274ca1"
server_key_iv: "bfe6554da1b4ce3ae097124b7b237612"
server_key_version: "encryptionkey@0"
I20260619 10:17:24.911120 4950 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781864224577058-825-0/minicluster-data/ts-0/wal/instance:
uuid: "4bcd6c8e17114dd8b3b4ea1f95dd5590"
format_stamp: "Formatted at 2026-06-19 10:17:24 on dist-test-slave-6pt2"
server_key: "4dcdb65acbdbba7ed853943fff274ca1"
server_key_iv: "bfe6554da1b4ce3ae097124b7b237612"
server_key_version: "encryptionkey@0"
I20260619 10:17:24.912452 4950 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 10:17:24.913146 4964 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 10:17:24.913326 4950 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 10:17:24.913417 4950 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781864224577058-825-0/minicluster-data/ts-0/data,/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781864224577058-825-0/minicluster-data/ts-0/wal
uuid: "4bcd6c8e17114dd8b3b4ea1f95dd5590"
format_stamp: "Formatted at 2026-06-19 10:17:24 on dist-test-slave-6pt2"
server_key: "4dcdb65acbdbba7ed853943fff274ca1"
server_key_iv: "bfe6554da1b4ce3ae097124b7b237612"
server_key_version: "encryptionkey@0"
I20260619 10:17:24.913484 4950 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781864224577058-825-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781864224577058-825-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781864224577058-825-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 10:17:24.927312 4950 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 10:17:24.929678 4950 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 10:17:24.929791 4950 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 10:17:24.930004 4950 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 10:17:24.930299 4950 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 10:17:24.930348 4950 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 10:17:24.930369 4950 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 10:17:24.930411 4950 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 10:17:24.936118 4950 rpc_server.cc:307] RPC server started. Bound to: 127.0.206.65:32933
I20260619 10:17:24.936184 5077 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.206.65:32933 every 8 connection(s)
I20260619 10:17:24.936432 4950 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781864224577058-825-0/minicluster-data/ts-0/data/info.pb
I20260619 10:17:24.941320 825 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTU9OP6/build/release/bin/kudu as pid 4950
I20260619 10:17:24.941419 825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781864224577058-825-0/minicluster-data/ts-0/wal/instance
I20260619 10:17:24.941601 825 external_mini_cluster.cc:1428] Setting key 67e79c70e1f19054f279be15d50d668b
Jun 19 10:17:24 dist-test-slave-6pt2 krb5kdc[4844](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864244, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.65@KRBTEST.COM for kudu/127.0.206.126@KRBTEST.COM
I20260619 10:17:24.946105 4927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:24.937159 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.65:48133 (local address 127.0.206.126:43221)
0619 10:17:24.937232 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0619 10:17:24.937234 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 10:17:24.937403 (+ 169us) server_negotiation.cc:407] Connection header received
0619 10:17:24.937835 (+ 432us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:17:24.937836 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:17:24.937845 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:17:24.937896 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:17:24.938828 (+ 932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:24.939322 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:24.940273 (+ 951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:24.940470 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:24.940528 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:24.942964 (+ 2436us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 10:17:24.942966 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 10:17:24.942966 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 10:17:24.942975 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0619 10:17:24.944389 (+ 1414us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:17:24.944932 (+ 543us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:17:24.944934 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:17:24.944934 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:17:24.945004 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:17:24.945298 (+ 294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:17:24.945313 (+ 15us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:17:24.945314 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:17:24.945459 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 10:17:24.945520 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0619 10:17:24.945869 (+ 349us) server_negotiation.cc:299] Negotiation successful
0619 10:17:24.945985 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260619 10:17:24.946308 5080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:24.937084 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:43221 (local address 127.0.206.65:48133)
0619 10:17:24.937273 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0619 10:17:24.937311 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0619 10:17:24.937756 (+ 445us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 10:17:24.937953 (+ 197us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 10:17:24.937956 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 10:17:24.938192 (+ 236us) client_negotiation.cc:189] Negotiated authn=SASL
0619 10:17:24.938751 (+ 559us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:24.938756 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:24.939417 (+ 661us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:24.939418 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:24.940183 (+ 765us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:24.940185 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:24.940544 (+ 359us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:24.940545 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:24.940639 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:24.941044 (+ 405us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 10:17:24.941051 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0619 10:17:24.942857 (+ 1806us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 10:17:24.944494 (+ 1637us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 10:17:24.944501 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 10:17:24.944503 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 10:17:24.944847 (+ 344us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 10:17:24.945087 (+ 240us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 10:17:24.945088 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 10:17:24.945088 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 10:17:24.945233 (+ 145us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 10:17:24.945533 (+ 300us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 10:17:24.945538 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 10:17:24.945701 (+ 163us) client_negotiation.cc:769] Sending connection context
0619 10:17:24.945849 (+ 148us) client_negotiation.cc:240] Negotiation successful
0619 10:17:24.946040 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":108,"threads_started":1}
I20260619 10:17:24.946861 5078 heartbeater.cc:347] Connected to a master server at 127.0.206.126:43221
I20260619 10:17:24.946976 5078 heartbeater.cc:464] Registering TS with master...
I20260619 10:17:24.947117 5078 heartbeater.cc:511] Master 127.0.206.126:43221 requested a full tablet report, sending...
I20260619 10:17:24.947618 4888 ts_manager.cc:194] Registered new tserver with Master: 4bcd6c8e17114dd8b3b4ea1f95dd5590 (127.0.206.65:32933)
I20260619 10:17:24.948212 4888 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.206.65@KRBTEST.COM'} at 127.0.206.65:48133
WARNING: no policy specified for kudu/127.0.206.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.206.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.206.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781864224577058-825-0/krb5kdc/kudu_127.0.206.66.keytab.
Entry for principal kudu/127.0.206.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781864224577058-825-0/krb5kdc/kudu_127.0.206.66.keytab.
I20260619 10:17:24.965198 825 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTU9OP6/build/release/bin/kudu
/tmp/dist-test-taskTU9OP6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781864224577058-825-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781864224577058-825-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781864224577058-825-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781864224577058-825-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.206.66:0
--local_ip_for_outbound_sockets=127.0.206.66
--webserver_interface=127.0.206.66
--webserver_port=0
--tserver_master_addrs=127.0.206.126:43221
--keytab_file=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781864224577058-825-0/krb5kdc/kudu_127.0.206.66.keytab
--principal=kudu/127.0.206.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.206.84:35871
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781864224577058-825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781864224577058-825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781864224577058-825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 10:17:25.043627 5083 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 10:17:25.043810 5083 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 10:17:25.043833 5083 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 10:17:25.044979 5083 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 10:17:25.045017 5083 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 10:17:25.045042 5083 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.206.66
I20260619 10:17:25.046506 5083 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.206.84:35871
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781864224577058-825-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781864224577058-825-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781864224577058-825-0/krb5kdc/kudu_127.0.206.66.keytab
--principal=kudu/127.0.206.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.206.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781864224577058-825-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.206.66
--webserver_port=0
--tserver_master_addrs=127.0.206.126:43221
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5083
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.206.66
--log_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781864224577058-825-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 09:32:44 UTC on ccycloud.quasar-gbtldp.root.comops.site
build id 4455440
I20260619 10:17:25.046746 5083 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 10:17:25.046944 5083 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 10:17:25.051472 5088 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 10:17:25.051498 5089 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 10:17:25.051573 5083 server_base.cc:1034] running on GCE node
W20260619 10:17:25.051493 5091 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 10:17:25.051870 5083 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 10:17:25.052098 5083 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 10:17:25.053246 5083 hybrid_clock.cc:648] HybridClock initialized: now 1781864245053236 us; error 26 us; skew 500 ppm
Jun 19 10:17:25 dist-test-slave-6pt2 krb5kdc[4844](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864245, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 10:17:25.055707 5083 init.cc:377] Logged in from keytab as kudu/127.0.206.66@KRBTEST.COM (short username kudu)
I20260619 10:17:25.056048 5083 fs_manager.cc:359] Metadata directory not provided
I20260619 10:17:25.056094 5083 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 10:17:25.056180 5083 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 10:17:25.057108 5083 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781864224577058-825-0/minicluster-data/ts-1/data/instance:
uuid: "269681e44b0645f09e2a3e7ef1465281"
format_stamp: "Formatted at 2026-06-19 10:17:25 on dist-test-slave-6pt2"
server_key: "5338a53db6f907827b5c0f10bb65453a"
server_key_iv: "556130ab5a204be5f598d13e04e4e68a"
server_key_version: "encryptionkey@0"
I20260619 10:17:25.057384 5083 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781864224577058-825-0/minicluster-data/ts-1/wal/instance:
uuid: "269681e44b0645f09e2a3e7ef1465281"
format_stamp: "Formatted at 2026-06-19 10:17:25 on dist-test-slave-6pt2"
server_key: "5338a53db6f907827b5c0f10bb65453a"
server_key_iv: "556130ab5a204be5f598d13e04e4e68a"
server_key_version: "encryptionkey@0"
I20260619 10:17:25.058516 5083 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 10:17:25.059231 5097 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 10:17:25.059422 5083 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260619 10:17:25.059489 5083 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781864224577058-825-0/minicluster-data/ts-1/data,/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781864224577058-825-0/minicluster-data/ts-1/wal
uuid: "269681e44b0645f09e2a3e7ef1465281"
format_stamp: "Formatted at 2026-06-19 10:17:25 on dist-test-slave-6pt2"
server_key: "5338a53db6f907827b5c0f10bb65453a"
server_key_iv: "556130ab5a204be5f598d13e04e4e68a"
server_key_version: "encryptionkey@0"
I20260619 10:17:25.059535 5083 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781864224577058-825-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781864224577058-825-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781864224577058-825-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 10:17:25.066802 5083 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 10:17:25.068861 5083 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 10:17:25.068979 5083 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 10:17:25.069170 5083 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 10:17:25.069586 5083 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 10:17:25.069620 5083 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 10:17:25.069640 5083 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 10:17:25.069680 5083 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 10:17:25.075292 5083 rpc_server.cc:307] RPC server started. Bound to: 127.0.206.66:46541
I20260619 10:17:25.075333 5210 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.206.66:46541 every 8 connection(s)
I20260619 10:17:25.075579 5083 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781864224577058-825-0/minicluster-data/ts-1/data/info.pb
I20260619 10:17:25.079293 825 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTU9OP6/build/release/bin/kudu as pid 5083
I20260619 10:17:25.079377 825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781864224577058-825-0/minicluster-data/ts-1/wal/instance
I20260619 10:17:25.079559 825 external_mini_cluster.cc:1428] Setting key 79128f179cd32da85176253a914f6f10
Jun 19 10:17:25 dist-test-slave-6pt2 krb5kdc[4844](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864245, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.66@KRBTEST.COM for kudu/127.0.206.126@KRBTEST.COM
I20260619 10:17:25.084980 4927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:25.076220 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.66:39519 (local address 127.0.206.126:43221)
0619 10:17:25.076282 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0619 10:17:25.076284 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 10:17:25.076461 (+ 177us) server_negotiation.cc:407] Connection header received
0619 10:17:25.076913 (+ 452us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:17:25.076914 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:17:25.076923 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:17:25.076981 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:17:25.077872 (+ 891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:25.078349 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:25.079467 (+ 1118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:25.079766 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:25.079864 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:25.082118 (+ 2254us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 10:17:25.082120 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 10:17:25.082120 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 10:17:25.082128 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 10:17:25.083631 (+ 1503us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:17:25.084035 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:17:25.084036 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:17:25.084037 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:17:25.084095 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:17:25.084340 (+ 245us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:17:25.084342 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:17:25.084342 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:17:25.084457 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 10:17:25.084532 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0619 10:17:25.084790 (+ 258us) server_negotiation.cc:299] Negotiation successful
0619 10:17:25.084880 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260619 10:17:25.085150 5213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:25.076185 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:43221 (local address 127.0.206.66:39519)
0619 10:17:25.076338 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0619 10:17:25.076377 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0619 10:17:25.076823 (+ 446us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 10:17:25.077051 (+ 228us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 10:17:25.077055 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 10:17:25.077319 (+ 264us) client_negotiation.cc:189] Negotiated authn=SASL
0619 10:17:25.077793 (+ 474us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:25.077798 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:25.078419 (+ 621us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:25.078420 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:25.079308 (+ 888us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:25.079310 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:25.079888 (+ 578us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:25.079889 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:25.080022 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:25.080561 (+ 539us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 10:17:25.080576 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0619 10:17:25.082014 (+ 1438us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 10:17:25.083721 (+ 1707us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 10:17:25.083728 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 10:17:25.083731 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 10:17:25.083947 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 10:17:25.084160 (+ 213us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 10:17:25.084161 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 10:17:25.084161 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 10:17:25.084268 (+ 107us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 10:17:25.084522 (+ 254us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 10:17:25.084528 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 10:17:25.084663 (+ 135us) client_negotiation.cc:769] Sending connection context
0619 10:17:25.084772 (+ 109us) client_negotiation.cc:240] Negotiation successful
0619 10:17:25.084920 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":91,"threads_started":1}
I20260619 10:17:25.085695 5211 heartbeater.cc:347] Connected to a master server at 127.0.206.126:43221
I20260619 10:17:25.085767 5211 heartbeater.cc:464] Registering TS with master...
I20260619 10:17:25.085915 5211 heartbeater.cc:511] Master 127.0.206.126:43221 requested a full tablet report, sending...
I20260619 10:17:25.086287 4888 ts_manager.cc:194] Registered new tserver with Master: 269681e44b0645f09e2a3e7ef1465281 (127.0.206.66:46541)
I20260619 10:17:25.086866 4888 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.206.66@KRBTEST.COM'} at 127.0.206.66:39519
WARNING: no policy specified for kudu/127.0.206.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.206.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.206.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781864224577058-825-0/krb5kdc/kudu_127.0.206.67.keytab.
Entry for principal kudu/127.0.206.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781864224577058-825-0/krb5kdc/kudu_127.0.206.67.keytab.
I20260619 10:17:25.103309 825 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTU9OP6/build/release/bin/kudu
/tmp/dist-test-taskTU9OP6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781864224577058-825-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781864224577058-825-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781864224577058-825-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781864224577058-825-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.206.67:0
--local_ip_for_outbound_sockets=127.0.206.67
--webserver_interface=127.0.206.67
--webserver_port=0
--tserver_master_addrs=127.0.206.126:43221
--keytab_file=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781864224577058-825-0/krb5kdc/kudu_127.0.206.67.keytab
--principal=kudu/127.0.206.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.206.84:35871
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781864224577058-825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781864224577058-825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781864224577058-825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 10:17:25.182809 5216 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 10:17:25.183008 5216 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 10:17:25.183032 5216 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 10:17:25.184132 5216 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 10:17:25.184167 5216 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 10:17:25.184192 5216 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.206.67
I20260619 10:17:25.185592 5216 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.206.84:35871
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781864224577058-825-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781864224577058-825-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781864224577058-825-0/krb5kdc/kudu_127.0.206.67.keytab
--principal=kudu/127.0.206.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.206.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781864224577058-825-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.206.67
--webserver_port=0
--tserver_master_addrs=127.0.206.126:43221
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5216
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.206.67
--log_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781864224577058-825-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 09:32:44 UTC on ccycloud.quasar-gbtldp.root.comops.site
build id 4455440
I20260619 10:17:25.185763 5216 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 10:17:25.185925 5216 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 10:17:25.190716 5221 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 10:17:25.190656 5224 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 10:17:25.190660 5222 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 10:17:25.190966 5216 server_base.cc:1034] running on GCE node
I20260619 10:17:25.191215 5216 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 10:17:25.191443 5216 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 10:17:25.192602 5216 hybrid_clock.cc:648] HybridClock initialized: now 1781864245192597 us; error 34 us; skew 500 ppm
Jun 19 10:17:25 dist-test-slave-6pt2 krb5kdc[4844](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864245, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 10:17:25.195021 5216 init.cc:377] Logged in from keytab as kudu/127.0.206.67@KRBTEST.COM (short username kudu)
I20260619 10:17:25.195346 5216 fs_manager.cc:359] Metadata directory not provided
I20260619 10:17:25.195417 5216 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 10:17:25.195573 5216 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 10:17:25.196581 5216 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781864224577058-825-0/minicluster-data/ts-2/data/instance:
uuid: "a391e464398446c89d091a8743089cca"
format_stamp: "Formatted at 2026-06-19 10:17:25 on dist-test-slave-6pt2"
server_key: "52da948e45511092c4fc91c88ee35146"
server_key_iv: "1a1b217efc233d7fdcbba8b60c3dc89e"
server_key_version: "encryptionkey@0"
I20260619 10:17:25.196939 5216 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781864224577058-825-0/minicluster-data/ts-2/wal/instance:
uuid: "a391e464398446c89d091a8743089cca"
format_stamp: "Formatted at 2026-06-19 10:17:25 on dist-test-slave-6pt2"
server_key: "52da948e45511092c4fc91c88ee35146"
server_key_iv: "1a1b217efc233d7fdcbba8b60c3dc89e"
server_key_version: "encryptionkey@0"
I20260619 10:17:25.198225 5216 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260619 10:17:25.198966 5230 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 10:17:25.199120 5216 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260619 10:17:25.199219 5216 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781864224577058-825-0/minicluster-data/ts-2/data,/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781864224577058-825-0/minicluster-data/ts-2/wal
uuid: "a391e464398446c89d091a8743089cca"
format_stamp: "Formatted at 2026-06-19 10:17:25 on dist-test-slave-6pt2"
server_key: "52da948e45511092c4fc91c88ee35146"
server_key_iv: "1a1b217efc233d7fdcbba8b60c3dc89e"
server_key_version: "encryptionkey@0"
I20260619 10:17:25.199290 5216 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781864224577058-825-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781864224577058-825-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781864224577058-825-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 10:17:25.211264 5216 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 10:17:25.213063 5216 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 10:17:25.213218 5216 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 10:17:25.213438 5216 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 10:17:25.213750 5216 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 10:17:25.213806 5216 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 10:17:25.213848 5216 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 10:17:25.213876 5216 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 10:17:25.219623 5216 rpc_server.cc:307] RPC server started. Bound to: 127.0.206.67:36495
I20260619 10:17:25.219666 5343 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.206.67:36495 every 8 connection(s)
I20260619 10:17:25.219928 5216 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781864224577058-825-0/minicluster-data/ts-2/data/info.pb
Jun 19 10:17:25 dist-test-slave-6pt2 krb5kdc[4844](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864245, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.67@KRBTEST.COM for kudu/127.0.206.126@KRBTEST.COM
I20260619 10:17:25.227802 825 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTU9OP6/build/release/bin/kudu as pid 5216
I20260619 10:17:25.227916 825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781864224577058-825-0/minicluster-data/ts-2/wal/instance
I20260619 10:17:25.228173 825 external_mini_cluster.cc:1428] Setting key 78f0bea46f7b3ab8eed6bbe2a4c97b6c
I20260619 10:17:25.229138 4927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:25.220606 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.67:47461 (local address 127.0.206.126:43221)
0619 10:17:25.220683 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0619 10:17:25.220685 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 10:17:25.220884 (+ 199us) server_negotiation.cc:407] Connection header received
0619 10:17:25.221330 (+ 446us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:17:25.221331 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:17:25.221339 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:17:25.221392 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:17:25.222275 (+ 883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:25.222768 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:25.223695 (+ 927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:25.223864 (+ 169us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:25.223916 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:25.226211 (+ 2295us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 10:17:25.226212 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 10:17:25.226212 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 10:17:25.226220 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 10:17:25.227414 (+ 1194us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:17:25.227844 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:17:25.227845 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:17:25.227846 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:17:25.227968 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:17:25.228308 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:17:25.228309 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:17:25.228309 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:17:25.228456 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 10:17:25.228539 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0619 10:17:25.228938 (+ 399us) server_negotiation.cc:299] Negotiation successful
0619 10:17:25.229017 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260619 10:17:25.229250 5346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:25.220588 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:43221 (local address 127.0.206.67:47461)
0619 10:17:25.220764 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0619 10:17:25.220801 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0619 10:17:25.221254 (+ 453us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 10:17:25.221446 (+ 192us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 10:17:25.221449 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 10:17:25.221709 (+ 260us) client_negotiation.cc:189] Negotiated authn=SASL
0619 10:17:25.222201 (+ 492us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:25.222206 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:25.222838 (+ 632us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:25.222839 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:25.223626 (+ 787us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:25.223628 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:25.223929 (+ 301us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:25.223930 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:25.224038 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:25.224696 (+ 658us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 10:17:25.224705 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0619 10:17:25.226116 (+ 1411us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 10:17:25.227514 (+ 1398us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 10:17:25.227520 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 10:17:25.227522 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 10:17:25.227755 (+ 233us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 10:17:25.228057 (+ 302us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 10:17:25.228059 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 10:17:25.228059 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 10:17:25.228205 (+ 146us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 10:17:25.228525 (+ 320us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 10:17:25.228531 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 10:17:25.228719 (+ 188us) client_negotiation.cc:769] Sending connection context
0619 10:17:25.228841 (+ 122us) client_negotiation.cc:240] Negotiation successful
0619 10:17:25.229024 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":87,"threads_started":1}
I20260619 10:17:25.229724 5344 heartbeater.cc:347] Connected to a master server at 127.0.206.126:43221
I20260619 10:17:25.229813 5344 heartbeater.cc:464] Registering TS with master...
I20260619 10:17:25.229921 5344 heartbeater.cc:511] Master 127.0.206.126:43221 requested a full tablet report, sending...
I20260619 10:17:25.230253 4888 ts_manager.cc:194] Registered new tserver with Master: a391e464398446c89d091a8743089cca (127.0.206.67:36495)
I20260619 10:17:25.230707 4888 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.206.67@KRBTEST.COM'} at 127.0.206.67:47461
I20260619 10:17:25.240140 825 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 10:17:25.247344 4927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:25.241417 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34700 (local address 127.0.206.126:43221)
0619 10:17:25.241476 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0619 10:17:25.241478 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 10:17:25.241620 (+ 142us) server_negotiation.cc:407] Connection header received
0619 10:17:25.241668 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:17:25.241668 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:17:25.241677 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:17:25.241723 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:17:25.242298 (+ 575us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:25.242817 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:25.243484 (+ 667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:25.243656 (+ 172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:25.243704 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:25.244823 (+ 1119us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 10:17:25.244824 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 10:17:25.244824 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 10:17:25.244830 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 10:17:25.246358 (+ 1528us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:17:25.246735 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:17:25.246736 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:17:25.246736 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:17:25.246783 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:17:25.246927 (+ 144us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:17:25.246928 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:17:25.246928 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:17:25.247053 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 10:17:25.247121 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0619 10:17:25.247206 (+ 85us) server_negotiation.cc:299] Negotiation successful
0619 10:17:25.247258 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260619 10:17:25.248929 4888 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34700:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 10:17:25.250487 4888 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 10:17:25.256872 5352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:25.252631 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.67:36495 (local address 127.0.0.1:56568)
0619 10:17:25.252801 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0619 10:17:25.252839 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0619 10:17:25.252898 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 10:17:25.253629 (+ 731us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 10:17:25.253629 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 10:17:25.253631 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 10:17:25.253667 (+ 36us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:25.253669 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:25.254523 (+ 854us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:25.254523 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:25.255831 (+ 1308us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:25.255833 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:25.256636 (+ 803us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:25.256637 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:25.256695 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:25.256699 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 10:17:25.256754 (+ 55us) client_negotiation.cc:240] Negotiation successful
0619 10:17:25.256794 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131,"mutex_wait_us":53,"thread_start_us":51,"threads_started":1}
I20260619 10:17:25.256883 5355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:25.252708 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56568 (local address 127.0.206.67:36495)
0619 10:17:25.253392 (+ 684us) server_negotiation.cc:206] Beginning negotiation
0619 10:17:25.253397 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 10:17:25.253405 (+ 8us) server_negotiation.cc:407] Connection header received
0619 10:17:25.253441 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:17:25.253443 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:17:25.253458 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:17:25.253543 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 10:17:25.253724 (+ 181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:25.254406 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:25.255908 (+ 1502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:25.256575 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:25.256637 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:25.256688 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0619 10:17:25.256768 (+ 80us) server_negotiation.cc:299] Negotiation successful
0619 10:17:25.256804 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":649,"thread_start_us":101,"threads_started":1}
I20260619 10:17:25.256976 5353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:25.252633 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.65:32933 (local address 127.0.0.1:33984)
0619 10:17:25.253031 (+ 398us) negotiation.cc:107] Waiting for socket to connect
0619 10:17:25.253041 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 10:17:25.253128 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 10:17:25.253318 (+ 190us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 10:17:25.253321 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 10:17:25.253323 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 10:17:25.253387 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:25.253389 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:25.254764 (+ 1375us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:25.254764 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:25.255657 (+ 893us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:25.255659 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:25.256540 (+ 881us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:25.256541 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:25.256614 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:25.256630 (+ 16us) client_negotiation.cc:769] Sending connection context
0619 10:17:25.256691 (+ 61us) client_negotiation.cc:240] Negotiation successful
0619 10:17:25.256745 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":385,"thread_start_us":52,"threads_started":1}
I20260619 10:17:25.257069 5354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:25.252646 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.66:46541 (local address 127.0.0.1:43534)
0619 10:17:25.253216 (+ 570us) negotiation.cc:107] Waiting for socket to connect
0619 10:17:25.253225 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0619 10:17:25.253272 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 10:17:25.253478 (+ 206us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 10:17:25.253479 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 10:17:25.253482 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 10:17:25.253536 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:25.253538 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:25.254852 (+ 1314us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:25.254852 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:25.255996 (+ 1144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:25.255998 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:25.256757 (+ 759us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:25.256758 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:25.256815 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:25.256819 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 10:17:25.256866 (+ 47us) client_negotiation.cc:240] Negotiation successful
0619 10:17:25.256897 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":556,"thread_start_us":76,"threads_started":1}
I20260619 10:17:25.257222 5356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:25.252878 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33984 (local address 127.0.206.65:32933)
0619 10:17:25.253047 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0619 10:17:25.253052 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 10:17:25.253120 (+ 68us) server_negotiation.cc:407] Connection header received
0619 10:17:25.253198 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:17:25.253201 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:17:25.253217 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:17:25.253302 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 10:17:25.253492 (+ 190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:25.254657 (+ 1165us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:25.255753 (+ 1096us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:25.256445 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:25.256520 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:25.256575 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0619 10:17:25.256837 (+ 262us) server_negotiation.cc:299] Negotiation successful
0619 10:17:25.256883 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":131,"thread_start_us":77,"threads_started":1}
I20260619 10:17:25.257301 5357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:25.252877 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43534 (local address 127.0.206.66:46541)
0619 10:17:25.253113 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0619 10:17:25.253115 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 10:17:25.253263 (+ 148us) server_negotiation.cc:407] Connection header received
0619 10:17:25.253335 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:17:25.253337 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:17:25.253352 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:17:25.253440 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 10:17:25.253717 (+ 277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:25.254764 (+ 1047us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:25.256079 (+ 1315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:25.256606 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:25.256673 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:25.256727 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0619 10:17:25.257175 (+ 448us) server_negotiation.cc:299] Negotiation successful
0619 10:17:25.257213 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":209,"thread_start_us":81,"threads_started":1}
I20260619 10:17:25.257622 5278 tablet_service.cc:1467] Processing CreateTablet for tablet b2929c3f4c944173aa5021d781a3e112 (DEFAULT_TABLE table=test-table [id=cfcf998a06244ff19f6a958059680be0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 10:17:25.257987 5278 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b2929c3f4c944173aa5021d781a3e112. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 10:17:25.257894 5012 tablet_service.cc:1467] Processing CreateTablet for tablet b2929c3f4c944173aa5021d781a3e112 (DEFAULT_TABLE table=test-table [id=cfcf998a06244ff19f6a958059680be0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 10:17:25.258070 5145 tablet_service.cc:1467] Processing CreateTablet for tablet b2929c3f4c944173aa5021d781a3e112 (DEFAULT_TABLE table=test-table [id=cfcf998a06244ff19f6a958059680be0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 10:17:25.258237 5012 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b2929c3f4c944173aa5021d781a3e112. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 10:17:25.258345 5145 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b2929c3f4c944173aa5021d781a3e112. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 10:17:25.260857 5358 tablet_bootstrap.cc:492] T b2929c3f4c944173aa5021d781a3e112 P 4bcd6c8e17114dd8b3b4ea1f95dd5590: Bootstrap starting.
I20260619 10:17:25.261253 5359 tablet_bootstrap.cc:492] T b2929c3f4c944173aa5021d781a3e112 P a391e464398446c89d091a8743089cca: Bootstrap starting.
I20260619 10:17:25.261507 5360 tablet_bootstrap.cc:492] T b2929c3f4c944173aa5021d781a3e112 P 269681e44b0645f09e2a3e7ef1465281: Bootstrap starting.
I20260619 10:17:25.261730 5358 tablet_bootstrap.cc:654] T b2929c3f4c944173aa5021d781a3e112 P 4bcd6c8e17114dd8b3b4ea1f95dd5590: Neither blocks nor log segments found. Creating new log.
I20260619 10:17:25.261988 5359 tablet_bootstrap.cc:654] T b2929c3f4c944173aa5021d781a3e112 P a391e464398446c89d091a8743089cca: Neither blocks nor log segments found. Creating new log.
I20260619 10:17:25.262169 5360 tablet_bootstrap.cc:654] T b2929c3f4c944173aa5021d781a3e112 P 269681e44b0645f09e2a3e7ef1465281: Neither blocks nor log segments found. Creating new log.
I20260619 10:17:25.262423 5358 log.cc:824] T b2929c3f4c944173aa5021d781a3e112 P 4bcd6c8e17114dd8b3b4ea1f95dd5590: Log is configured to *not* fsync() on all Append() calls
I20260619 10:17:25.262593 5359 log.cc:824] T b2929c3f4c944173aa5021d781a3e112 P a391e464398446c89d091a8743089cca: Log is configured to *not* fsync() on all Append() calls
I20260619 10:17:25.262799 5360 log.cc:824] T b2929c3f4c944173aa5021d781a3e112 P 269681e44b0645f09e2a3e7ef1465281: Log is configured to *not* fsync() on all Append() calls
I20260619 10:17:25.263245 5358 tablet_bootstrap.cc:492] T b2929c3f4c944173aa5021d781a3e112 P 4bcd6c8e17114dd8b3b4ea1f95dd5590: No bootstrap required, opened a new log
I20260619 10:17:25.263276 5359 tablet_bootstrap.cc:492] T b2929c3f4c944173aa5021d781a3e112 P a391e464398446c89d091a8743089cca: No bootstrap required, opened a new log
I20260619 10:17:25.263319 5359 ts_tablet_manager.cc:1397] T b2929c3f4c944173aa5021d781a3e112 P a391e464398446c89d091a8743089cca: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260619 10:17:25.263311 5358 ts_tablet_manager.cc:1397] T b2929c3f4c944173aa5021d781a3e112 P 4bcd6c8e17114dd8b3b4ea1f95dd5590: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.001s
I20260619 10:17:25.263411 5360 tablet_bootstrap.cc:492] T b2929c3f4c944173aa5021d781a3e112 P 269681e44b0645f09e2a3e7ef1465281: No bootstrap required, opened a new log
I20260619 10:17:25.263458 5360 ts_tablet_manager.cc:1397] T b2929c3f4c944173aa5021d781a3e112 P 269681e44b0645f09e2a3e7ef1465281: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260619 10:17:25.264685 5360 raft_consensus.cc:348] T b2929c3f4c944173aa5021d781a3e112 P 269681e44b0645f09e2a3e7ef1465281 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bcd6c8e17114dd8b3b4ea1f95dd5590" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 32933 } } peers { permanent_uuid: "a391e464398446c89d091a8743089cca" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 36495 } } peers { permanent_uuid: "269681e44b0645f09e2a3e7ef1465281" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 46541 } }
I20260619 10:17:25.264813 5360 raft_consensus.cc:374] T b2929c3f4c944173aa5021d781a3e112 P 269681e44b0645f09e2a3e7ef1465281 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 10:17:25.264833 5360 raft_consensus.cc:729] T b2929c3f4c944173aa5021d781a3e112 P 269681e44b0645f09e2a3e7ef1465281 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 269681e44b0645f09e2a3e7ef1465281, State: Initialized, Role: FOLLOWER
I20260619 10:17:25.264950 5360 consensus_queue.cc:260] T b2929c3f4c944173aa5021d781a3e112 P 269681e44b0645f09e2a3e7ef1465281 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bcd6c8e17114dd8b3b4ea1f95dd5590" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 32933 } } peers { permanent_uuid: "a391e464398446c89d091a8743089cca" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 36495 } } peers { permanent_uuid: "269681e44b0645f09e2a3e7ef1465281" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 46541 } }
I20260619 10:17:25.265213 5360 ts_tablet_manager.cc:1428] T b2929c3f4c944173aa5021d781a3e112 P 269681e44b0645f09e2a3e7ef1465281: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260619 10:17:25.265344 5211 heartbeater.cc:503] Master 127.0.206.126:43221 was elected leader, sending a full tablet report...
I20260619 10:17:25.265272 5358 raft_consensus.cc:348] T b2929c3f4c944173aa5021d781a3e112 P 4bcd6c8e17114dd8b3b4ea1f95dd5590 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bcd6c8e17114dd8b3b4ea1f95dd5590" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 32933 } } peers { permanent_uuid: "a391e464398446c89d091a8743089cca" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 36495 } } peers { permanent_uuid: "269681e44b0645f09e2a3e7ef1465281" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 46541 } }
I20260619 10:17:25.265300 5359 raft_consensus.cc:348] T b2929c3f4c944173aa5021d781a3e112 P a391e464398446c89d091a8743089cca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bcd6c8e17114dd8b3b4ea1f95dd5590" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 32933 } } peers { permanent_uuid: "a391e464398446c89d091a8743089cca" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 36495 } } peers { permanent_uuid: "269681e44b0645f09e2a3e7ef1465281" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 46541 } }
I20260619 10:17:25.265419 5358 raft_consensus.cc:374] T b2929c3f4c944173aa5021d781a3e112 P 4bcd6c8e17114dd8b3b4ea1f95dd5590 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 10:17:25.265420 5359 raft_consensus.cc:374] T b2929c3f4c944173aa5021d781a3e112 P a391e464398446c89d091a8743089cca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 10:17:25.265453 5359 raft_consensus.cc:729] T b2929c3f4c944173aa5021d781a3e112 P a391e464398446c89d091a8743089cca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a391e464398446c89d091a8743089cca, State: Initialized, Role: FOLLOWER
I20260619 10:17:25.265453 5358 raft_consensus.cc:729] T b2929c3f4c944173aa5021d781a3e112 P 4bcd6c8e17114dd8b3b4ea1f95dd5590 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4bcd6c8e17114dd8b3b4ea1f95dd5590, State: Initialized, Role: FOLLOWER
I20260619 10:17:25.265565 5358 consensus_queue.cc:260] T b2929c3f4c944173aa5021d781a3e112 P 4bcd6c8e17114dd8b3b4ea1f95dd5590 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bcd6c8e17114dd8b3b4ea1f95dd5590" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 32933 } } peers { permanent_uuid: "a391e464398446c89d091a8743089cca" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 36495 } } peers { permanent_uuid: "269681e44b0645f09e2a3e7ef1465281" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 46541 } }
I20260619 10:17:25.265570 5359 consensus_queue.cc:260] T b2929c3f4c944173aa5021d781a3e112 P a391e464398446c89d091a8743089cca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bcd6c8e17114dd8b3b4ea1f95dd5590" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 32933 } } peers { permanent_uuid: "a391e464398446c89d091a8743089cca" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 36495 } } peers { permanent_uuid: "269681e44b0645f09e2a3e7ef1465281" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 46541 } }
I20260619 10:17:25.265815 5359 ts_tablet_manager.cc:1428] T b2929c3f4c944173aa5021d781a3e112 P a391e464398446c89d091a8743089cca: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260619 10:17:25.265815 5358 ts_tablet_manager.cc:1428] T b2929c3f4c944173aa5021d781a3e112 P 4bcd6c8e17114dd8b3b4ea1f95dd5590: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260619 10:17:25.265823 5078 heartbeater.cc:503] Master 127.0.206.126:43221 was elected leader, sending a full tablet report...
I20260619 10:17:25.265940 5344 heartbeater.cc:503] Master 127.0.206.126:43221 was elected leader, sending a full tablet report...
I20260619 10:17:25.283058 5366 raft_consensus.cc:482] T b2929c3f4c944173aa5021d781a3e112 P 4bcd6c8e17114dd8b3b4ea1f95dd5590 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 10:17:25.283145 5366 raft_consensus.cc:504] T b2929c3f4c944173aa5021d781a3e112 P 4bcd6c8e17114dd8b3b4ea1f95dd5590 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bcd6c8e17114dd8b3b4ea1f95dd5590" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 32933 } } peers { permanent_uuid: "a391e464398446c89d091a8743089cca" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 36495 } } peers { permanent_uuid: "269681e44b0645f09e2a3e7ef1465281" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 46541 } }
I20260619 10:17:25.283437 5366 leader_election.cc:288] T b2929c3f4c944173aa5021d781a3e112 P 4bcd6c8e17114dd8b3b4ea1f95dd5590 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a391e464398446c89d091a8743089cca (127.0.206.67:36495), 269681e44b0645f09e2a3e7ef1465281 (127.0.206.66:46541)
I20260619 10:17:25.286234 5355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:25.283675 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.65:33845 (local address 127.0.206.67:36495)
0619 10:17:25.283718 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0619 10:17:25.283720 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 10:17:25.283730 (+ 10us) server_negotiation.cc:407] Connection header received
0619 10:17:25.283754 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:17:25.283755 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:17:25.283766 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:17:25.283822 (+ 56us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 10:17:25.283991 (+ 169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:25.284486 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:25.285524 (+ 1038us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:25.285878 (+ 354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:25.285961 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:25.285971 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0619 10:17:25.286135 (+ 164us) server_negotiation.cc:299] Negotiation successful
0619 10:17:25.286170 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260619 10:17:25.286237 5357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:25.283659 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.65:57391 (local address 127.0.206.66:46541)
0619 10:17:25.283698 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0619 10:17:25.283700 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 10:17:25.283827 (+ 127us) server_negotiation.cc:407] Connection header received
0619 10:17:25.283852 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:17:25.283853 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:17:25.283862 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:17:25.283921 (+ 59us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 10:17:25.284102 (+ 181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:25.284562 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:25.285528 (+ 966us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:25.285880 (+ 352us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:25.285967 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:25.285975 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0619 10:17:25.286135 (+ 160us) server_negotiation.cc:299] Negotiation successful
0619 10:17:25.286168 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260619 10:17:25.286260 5080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:25.283574 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.67:36495 (local address 127.0.206.65:33845)
0619 10:17:25.283639 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0619 10:17:25.283651 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 10:17:25.283695 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 10:17:25.283836 (+ 141us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 10:17:25.283837 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 10:17:25.283839 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 10:17:25.283912 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:25.283914 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:25.284557 (+ 643us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:25.284558 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:25.285411 (+ 853us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:25.285413 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:25.285953 (+ 540us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:25.285954 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:25.286040 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:25.286045 (+ 5us) client_negotiation.cc:769] Sending connection context
0619 10:17:25.286128 (+ 83us) client_negotiation.cc:240] Negotiation successful
0619 10:17:25.286169 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":43,"mutex_wait_us":44}
I20260619 10:17:25.286376 5367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:25.283579 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.66:46541 (local address 127.0.206.65:57391)
0619 10:17:25.283733 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0619 10:17:25.283744 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 10:17:25.283803 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 10:17:25.283935 (+ 132us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 10:17:25.283936 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 10:17:25.283937 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 10:17:25.284032 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:25.284033 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:25.284637 (+ 604us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:25.284638 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:25.285412 (+ 774us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:25.285414 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:25.285980 (+ 566us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:25.285981 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:25.286048 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:25.286051 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 10:17:25.286110 (+ 59us) client_negotiation.cc:240] Negotiation successful
0619 10:17:25.286154 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":137,"mutex_wait_us":13,"thread_start_us":83,"threads_started":1}
I20260619 10:17:25.286564 5298 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b2929c3f4c944173aa5021d781a3e112" candidate_uuid: "4bcd6c8e17114dd8b3b4ea1f95dd5590" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a391e464398446c89d091a8743089cca" is_pre_election: true
I20260619 10:17:25.286715 5298 raft_consensus.cc:2454] T b2929c3f4c944173aa5021d781a3e112 P a391e464398446c89d091a8743089cca [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4bcd6c8e17114dd8b3b4ea1f95dd5590 in term 0.
I20260619 10:17:25.286747 5165 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b2929c3f4c944173aa5021d781a3e112" candidate_uuid: "4bcd6c8e17114dd8b3b4ea1f95dd5590" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "269681e44b0645f09e2a3e7ef1465281" is_pre_election: true
I20260619 10:17:25.286901 5165 raft_consensus.cc:2454] T b2929c3f4c944173aa5021d781a3e112 P 269681e44b0645f09e2a3e7ef1465281 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4bcd6c8e17114dd8b3b4ea1f95dd5590 in term 0.
I20260619 10:17:25.286898 4966 leader_election.cc:302] T b2929c3f4c944173aa5021d781a3e112 P 4bcd6c8e17114dd8b3b4ea1f95dd5590 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4bcd6c8e17114dd8b3b4ea1f95dd5590, a391e464398446c89d091a8743089cca; no voters:
I20260619 10:17:25.287014 5366 raft_consensus.cc:2781] T b2929c3f4c944173aa5021d781a3e112 P 4bcd6c8e17114dd8b3b4ea1f95dd5590 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 10:17:25.287078 5366 raft_consensus.cc:482] T b2929c3f4c944173aa5021d781a3e112 P 4bcd6c8e17114dd8b3b4ea1f95dd5590 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 10:17:25.287101 5366 raft_consensus.cc:3037] T b2929c3f4c944173aa5021d781a3e112 P 4bcd6c8e17114dd8b3b4ea1f95dd5590 [term 0 FOLLOWER]: Advancing to term 1
I20260619 10:17:25.287729 5366 raft_consensus.cc:504] T b2929c3f4c944173aa5021d781a3e112 P 4bcd6c8e17114dd8b3b4ea1f95dd5590 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bcd6c8e17114dd8b3b4ea1f95dd5590" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 32933 } } peers { permanent_uuid: "a391e464398446c89d091a8743089cca" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 36495 } } peers { permanent_uuid: "269681e44b0645f09e2a3e7ef1465281" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 46541 } }
I20260619 10:17:25.287876 5366 leader_election.cc:288] T b2929c3f4c944173aa5021d781a3e112 P 4bcd6c8e17114dd8b3b4ea1f95dd5590 [CANDIDATE]: Term 1 election: Requested vote from peers a391e464398446c89d091a8743089cca (127.0.206.67:36495), 269681e44b0645f09e2a3e7ef1465281 (127.0.206.66:46541)
I20260619 10:17:25.288015 5165 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b2929c3f4c944173aa5021d781a3e112" candidate_uuid: "4bcd6c8e17114dd8b3b4ea1f95dd5590" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "269681e44b0645f09e2a3e7ef1465281"
I20260619 10:17:25.288081 5165 raft_consensus.cc:3037] T b2929c3f4c944173aa5021d781a3e112 P 269681e44b0645f09e2a3e7ef1465281 [term 0 FOLLOWER]: Advancing to term 1
I20260619 10:17:25.288045 5298 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b2929c3f4c944173aa5021d781a3e112" candidate_uuid: "4bcd6c8e17114dd8b3b4ea1f95dd5590" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a391e464398446c89d091a8743089cca"
I20260619 10:17:25.288146 5298 raft_consensus.cc:3037] T b2929c3f4c944173aa5021d781a3e112 P a391e464398446c89d091a8743089cca [term 0 FOLLOWER]: Advancing to term 1
I20260619 10:17:25.288803 5165 raft_consensus.cc:2454] T b2929c3f4c944173aa5021d781a3e112 P 269681e44b0645f09e2a3e7ef1465281 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4bcd6c8e17114dd8b3b4ea1f95dd5590 in term 1.
I20260619 10:17:25.288932 5298 raft_consensus.cc:2454] T b2929c3f4c944173aa5021d781a3e112 P a391e464398446c89d091a8743089cca [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4bcd6c8e17114dd8b3b4ea1f95dd5590 in term 1.
I20260619 10:17:25.288956 4965 leader_election.cc:302] T b2929c3f4c944173aa5021d781a3e112 P 4bcd6c8e17114dd8b3b4ea1f95dd5590 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 269681e44b0645f09e2a3e7ef1465281, 4bcd6c8e17114dd8b3b4ea1f95dd5590; no voters:
I20260619 10:17:25.289058 5366 raft_consensus.cc:2781] T b2929c3f4c944173aa5021d781a3e112 P 4bcd6c8e17114dd8b3b4ea1f95dd5590 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 10:17:25.289212 5366 raft_consensus.cc:686] T b2929c3f4c944173aa5021d781a3e112 P 4bcd6c8e17114dd8b3b4ea1f95dd5590 [term 1 LEADER]: Becoming Leader. State: Replica: 4bcd6c8e17114dd8b3b4ea1f95dd5590, State: Running, Role: LEADER
I20260619 10:17:25.289426 5366 consensus_queue.cc:237] T b2929c3f4c944173aa5021d781a3e112 P 4bcd6c8e17114dd8b3b4ea1f95dd5590 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bcd6c8e17114dd8b3b4ea1f95dd5590" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 32933 } } peers { permanent_uuid: "a391e464398446c89d091a8743089cca" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 36495 } } peers { permanent_uuid: "269681e44b0645f09e2a3e7ef1465281" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 46541 } }
I20260619 10:17:25.290213 4887 catalog_manager.cc:5495] T b2929c3f4c944173aa5021d781a3e112 P 4bcd6c8e17114dd8b3b4ea1f95dd5590 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4bcd6c8e17114dd8b3b4ea1f95dd5590 (127.0.206.65). New cstate: current_term: 1 leader_uuid: "4bcd6c8e17114dd8b3b4ea1f95dd5590" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4bcd6c8e17114dd8b3b4ea1f95dd5590" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 32933 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a391e464398446c89d091a8743089cca" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 36495 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "269681e44b0645f09e2a3e7ef1465281" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 46541 } health_report { overall_health: UNKNOWN } } }
I20260619 10:17:25.302623 5356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:25.300439 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33990 (local address 127.0.206.65:32933)
0619 10:17:25.300495 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0619 10:17:25.300497 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 10:17:25.300504 (+ 7us) server_negotiation.cc:407] Connection header received
0619 10:17:25.300540 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:17:25.300541 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:17:25.300555 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:17:25.300609 (+ 54us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 10:17:25.300728 (+ 119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:25.301161 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:25.301895 (+ 734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:25.302116 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:25.302169 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:25.302313 (+ 144us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 10:17:25.302398 (+ 85us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 10:17:25.302457 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0619 10:17:25.302487 (+ 30us) server_negotiation.cc:299] Negotiation successful
0619 10:17:25.302504 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260619 10:17:25.304260 5298 raft_consensus.cc:1261] T b2929c3f4c944173aa5021d781a3e112 P a391e464398446c89d091a8743089cca [term 1 FOLLOWER]: Refusing update from remote peer 4bcd6c8e17114dd8b3b4ea1f95dd5590: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 10:17:25.304260 5165 raft_consensus.cc:1261] T b2929c3f4c944173aa5021d781a3e112 P 269681e44b0645f09e2a3e7ef1465281 [term 1 FOLLOWER]: Refusing update from remote peer 4bcd6c8e17114dd8b3b4ea1f95dd5590: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 10:17:25.304605 5368 consensus_queue.cc:1035] T b2929c3f4c944173aa5021d781a3e112 P 4bcd6c8e17114dd8b3b4ea1f95dd5590 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a391e464398446c89d091a8743089cca" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 36495 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 10:17:25.304752 5366 consensus_queue.cc:1035] T b2929c3f4c944173aa5021d781a3e112 P 4bcd6c8e17114dd8b3b4ea1f95dd5590 [LEADER]: Connected to new peer: Peer: permanent_uuid: "269681e44b0645f09e2a3e7ef1465281" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 46541 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 10:17:25.307355 5373 mvcc.cc:205] Tried to move back new op lower bound from 7298515948763549696 to 7298515948706070528. Current Snapshot: MvccSnapshot[applied={T|T < 7298515948763549696}]
I20260619 10:17:25.309638 4887 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34700:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 10:17:25.309741 4887 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34700:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 10:17:25.310461 4887 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 221a9ce12eb94059a9fcb57a2fd812fc: Sending DeleteTablet for 3 replicas of tablet b2929c3f4c944173aa5021d781a3e112
I20260619 10:17:25.310765 5012 tablet_service.cc:1514] Processing DeleteTablet for tablet b2929c3f4c944173aa5021d781a3e112 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 10:17:25 UTC) from {username='kudu', principal='kudu/127.0.206.126@KRBTEST.COM'} at 127.0.0.1:33984
I20260619 10:17:25.310827 5278 tablet_service.cc:1514] Processing DeleteTablet for tablet b2929c3f4c944173aa5021d781a3e112 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 10:17:25 UTC) from {username='kudu', principal='kudu/127.0.206.126@KRBTEST.COM'} at 127.0.0.1:56568
I20260619 10:17:25.311002 5381 tablet_replica.cc:330] T b2929c3f4c944173aa5021d781a3e112 P 4bcd6c8e17114dd8b3b4ea1f95dd5590: stopping tablet replica
I20260619 10:17:25.311012 5145 tablet_service.cc:1514] Processing DeleteTablet for tablet b2929c3f4c944173aa5021d781a3e112 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 10:17:25 UTC) from {username='kudu', principal='kudu/127.0.206.126@KRBTEST.COM'} at 127.0.0.1:43534
I20260619 10:17:25.311128 5381 raft_consensus.cc:2229] T b2929c3f4c944173aa5021d781a3e112 P 4bcd6c8e17114dd8b3b4ea1f95dd5590 [term 1 LEADER]: Raft consensus shutting down.
I20260619 10:17:25.311256 5381 raft_consensus.cc:2258] T b2929c3f4c944173aa5021d781a3e112 P 4bcd6c8e17114dd8b3b4ea1f95dd5590 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 10:17:25.311275 5383 tablet_replica.cc:330] T b2929c3f4c944173aa5021d781a3e112 P 269681e44b0645f09e2a3e7ef1465281: stopping tablet replica
I20260619 10:17:25.311444 5383 raft_consensus.cc:2229] T b2929c3f4c944173aa5021d781a3e112 P 269681e44b0645f09e2a3e7ef1465281 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 10:17:25.311467 825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTU9OP6/build/release/bin/kudu with pid 4950
I20260619 10:17:25.311597 5383 raft_consensus.cc:2258] T b2929c3f4c944173aa5021d781a3e112 P 269681e44b0645f09e2a3e7ef1465281 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 10:17:25.312471 5383 ts_tablet_manager.cc:1905] T b2929c3f4c944173aa5021d781a3e112 P 269681e44b0645f09e2a3e7ef1465281: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 10:17:25.312902 5382 tablet_replica.cc:330] T b2929c3f4c944173aa5021d781a3e112 P a391e464398446c89d091a8743089cca: stopping tablet replica
I20260619 10:17:25.313011 5382 raft_consensus.cc:2229] T b2929c3f4c944173aa5021d781a3e112 P a391e464398446c89d091a8743089cca [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 10:17:25.313129 5382 raft_consensus.cc:2258] T b2929c3f4c944173aa5021d781a3e112 P a391e464398446c89d091a8743089cca [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 10:17:25.313665 5382 ts_tablet_manager.cc:1905] T b2929c3f4c944173aa5021d781a3e112 P a391e464398446c89d091a8743089cca: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 10:17:25.314468 5383 ts_tablet_manager.cc:1918] T b2929c3f4c944173aa5021d781a3e112 P 269681e44b0645f09e2a3e7ef1465281: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 10:17:25.314556 5383 log.cc:1196] T b2929c3f4c944173aa5021d781a3e112 P 269681e44b0645f09e2a3e7ef1465281: Deleting WAL directory at /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781864224577058-825-0/minicluster-data/ts-1/wal/wals/b2929c3f4c944173aa5021d781a3e112
I20260619 10:17:25.314841 5383 ts_tablet_manager.cc:1939] T b2929c3f4c944173aa5021d781a3e112 P 269681e44b0645f09e2a3e7ef1465281: Deleting consensus metadata
I20260619 10:17:25.315227 4872 catalog_manager.cc:4841] TS 269681e44b0645f09e2a3e7ef1465281 (127.0.206.66:46541): tablet b2929c3f4c944173aa5021d781a3e112 (table test-table [id=cfcf998a06244ff19f6a958059680be0]) successfully deleted
I20260619 10:17:25.315546 5382 ts_tablet_manager.cc:1918] T b2929c3f4c944173aa5021d781a3e112 P a391e464398446c89d091a8743089cca: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 10:17:25.315618 5382 log.cc:1196] T b2929c3f4c944173aa5021d781a3e112 P a391e464398446c89d091a8743089cca: Deleting WAL directory at /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781864224577058-825-0/minicluster-data/ts-2/wal/wals/b2929c3f4c944173aa5021d781a3e112
I20260619 10:17:25.315883 5382 ts_tablet_manager.cc:1939] T b2929c3f4c944173aa5021d781a3e112 P a391e464398446c89d091a8743089cca: Deleting consensus metadata
I20260619 10:17:25.316282 4873 catalog_manager.cc:4841] TS a391e464398446c89d091a8743089cca (127.0.206.67:36495): tablet b2929c3f4c944173aa5021d781a3e112 (table test-table [id=cfcf998a06244ff19f6a958059680be0]) successfully deleted
W20260619 10:17:25.317492 4875 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.0.206.65:32933 (error 108)
I20260619 10:17:25.317701 825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTU9OP6/build/release/bin/kudu with pid 5083
I20260619 10:17:25.318226 5354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:25.317787 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.65:32933 (local address 127.0.0.1:33992)
0619 10:17:25.318155 (+ 368us) negotiation.cc:107] Waiting for socket to connect
0619 10:17:25.318189 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.65:32933: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":347}
W20260619 10:17:25.318506 4875 catalog_manager.cc:4584] TS 4bcd6c8e17114dd8b3b4ea1f95dd5590 (127.0.206.65:32933): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet b2929c3f4c944173aa5021d781a3e112: Network error: Client connection negotiation failed: client connection to 127.0.206.65:32933: connect: Connection refused (error 111)
I20260619 10:17:25.322361 825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTU9OP6/build/release/bin/kudu with pid 5216
I20260619 10:17:25.326327 825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTU9OP6/build/release/bin/kudu with pid 4857
2026-06-19T10:17:25Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (1827 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-06-19T10:17:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T10:17:25Z Disabled control of system clock
I20260619 10:17:25.353057 825 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTU9OP6/build/release/bin/kudu
/tmp/dist-test-taskTU9OP6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781864224577058-825-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781864224577058-825-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781864224577058-825-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781864224577058-825-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.206.126:40327
--webserver_interface=127.0.206.126
--webserver_port=0
--builtin_ntp_servers=127.0.206.84:41839
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.206.126:40327
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260619 10:17:25.431543 5388 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 10:17:25.431733 5388 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 10:17:25.431754 5388 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 10:17:25.432835 5388 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 10:17:25.432871 5388 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 10:17:25.432884 5388 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 10:17:25.432897 5388 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 10:17:25.432909 5388 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 10:17:25.434159 5388 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.206.84:41839
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781864224577058-825-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781864224577058-825-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.206.126:40327
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.206.126:40327
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781864224577058-825-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.206.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5388
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781864224577058-825-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 09:32:44 UTC on ccycloud.quasar-gbtldp.root.comops.site
build id 4455440
I20260619 10:17:25.434333 5388 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 10:17:25.434553 5388 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 10:17:25.439141 5394 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 10:17:25.439126 5396 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 10:17:25.439218 5388 server_base.cc:1034] running on GCE node
W20260619 10:17:25.439126 5393 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 10:17:25.439632 5388 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 10:17:25.439927 5388 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 10:17:25.441066 5388 hybrid_clock.cc:648] HybridClock initialized: now 1781864245441057 us; error 28 us; skew 500 ppm
I20260619 10:17:25.442214 5388 webserver.cc:460] Webserver started at http://127.0.206.126:44445/ using document root <none> and password file <none>
I20260619 10:17:25.442466 5388 fs_manager.cc:359] Metadata directory not provided
I20260619 10:17:25.442510 5388 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 10:17:25.442723 5388 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 10:17:25.443763 5388 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781864224577058-825-0/minicluster-data/master-0/data/instance:
uuid: "ff3e5dd931fb408ab774a9ac881109ae"
format_stamp: "Formatted at 2026-06-19 10:17:25 on dist-test-slave-6pt2"
server_key: "e41cc1949e34929a057ce77238a2e082"
server_key_iv: "1d5809451f8e1dc6f0e8e153d06a5297"
server_key_version: "encryptionkey@0"
I20260619 10:17:25.444123 5388 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781864224577058-825-0/minicluster-data/master-0/wal/instance:
uuid: "ff3e5dd931fb408ab774a9ac881109ae"
format_stamp: "Formatted at 2026-06-19 10:17:25 on dist-test-slave-6pt2"
server_key: "e41cc1949e34929a057ce77238a2e082"
server_key_iv: "1d5809451f8e1dc6f0e8e153d06a5297"
server_key_version: "encryptionkey@0"
I20260619 10:17:25.445432 5388 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 10:17:25.446148 5402 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 10:17:25.446343 5388 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 10:17:25.446470 5388 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781864224577058-825-0/minicluster-data/master-0/data,/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781864224577058-825-0/minicluster-data/master-0/wal
uuid: "ff3e5dd931fb408ab774a9ac881109ae"
format_stamp: "Formatted at 2026-06-19 10:17:25 on dist-test-slave-6pt2"
server_key: "e41cc1949e34929a057ce77238a2e082"
server_key_iv: "1d5809451f8e1dc6f0e8e153d06a5297"
server_key_version: "encryptionkey@0"
I20260619 10:17:25.446539 5388 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781864224577058-825-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781864224577058-825-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781864224577058-825-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 10:17:25.449703 5388 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 10:17:25.449939 5388 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 10:17:25.450088 5388 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 10:17:25.453804 5388 rpc_server.cc:307] RPC server started. Bound to: 127.0.206.126:40327
I20260619 10:17:25.453830 5454 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.206.126:40327 every 8 connection(s)
I20260619 10:17:25.454149 5388 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781864224577058-825-0/minicluster-data/master-0/data/info.pb
I20260619 10:17:25.454766 5455 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 10:17:25.457046 825 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTU9OP6/build/release/bin/kudu as pid 5388
I20260619 10:17:25.457135 825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781864224577058-825-0/minicluster-data/master-0/wal/instance
I20260619 10:17:25.457309 825 external_mini_cluster.cc:1428] Setting key ce36ebbeb41eb8b02f56cd581288caa8
I20260619 10:17:25.457562 5455 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ff3e5dd931fb408ab774a9ac881109ae: Bootstrap starting.
I20260619 10:17:25.458186 5455 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ff3e5dd931fb408ab774a9ac881109ae: Neither blocks nor log segments found. Creating new log.
I20260619 10:17:25.458460 5455 log.cc:824] T 00000000000000000000000000000000 P ff3e5dd931fb408ab774a9ac881109ae: Log is configured to *not* fsync() on all Append() calls
I20260619 10:17:25.459419 5455 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ff3e5dd931fb408ab774a9ac881109ae: No bootstrap required, opened a new log
I20260619 10:17:25.459767 5458 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:25.457810 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36140 (local address 127.0.206.126:40327)
0619 10:17:25.457994 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0619 10:17:25.458005 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0619 10:17:25.458033 (+ 28us) server_negotiation.cc:407] Connection header received
0619 10:17:25.458488 (+ 455us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:17:25.458495 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:17:25.458532 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:17:25.458756 (+ 224us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:17:25.459328 (+ 572us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 10:17:25.459331 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 10:17:25.459332 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 10:17:25.459336 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0619 10:17:25.459386 (+ 50us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 10:17:25.459396 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 10:17:25.459489 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0619 10:17:25.459526 (+ 37us) server_negotiation.cc:299] Negotiation successful
0619 10:17:25.459593 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":116,"thread_start_us":91,"threads_started":1}
I20260619 10:17:25.460848 5455 raft_consensus.cc:348] T 00000000000000000000000000000000 P ff3e5dd931fb408ab774a9ac881109ae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ff3e5dd931fb408ab774a9ac881109ae" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 40327 } }
I20260619 10:17:25.461014 5455 raft_consensus.cc:374] T 00000000000000000000000000000000 P ff3e5dd931fb408ab774a9ac881109ae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 10:17:25.461071 5455 raft_consensus.cc:729] T 00000000000000000000000000000000 P ff3e5dd931fb408ab774a9ac881109ae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ff3e5dd931fb408ab774a9ac881109ae, State: Initialized, Role: FOLLOWER
I20260619 10:17:25.461216 5455 consensus_queue.cc:260] T 00000000000000000000000000000000 P ff3e5dd931fb408ab774a9ac881109ae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ff3e5dd931fb408ab774a9ac881109ae" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 40327 } }
I20260619 10:17:25.461314 5455 raft_consensus.cc:388] T 00000000000000000000000000000000 P ff3e5dd931fb408ab774a9ac881109ae [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 10:17:25.461359 5455 raft_consensus.cc:482] T 00000000000000000000000000000000 P ff3e5dd931fb408ab774a9ac881109ae [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 10:17:25.461405 5455 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ff3e5dd931fb408ab774a9ac881109ae [term 0 FOLLOWER]: Advancing to term 1
I20260619 10:17:25.462131 5455 raft_consensus.cc:504] T 00000000000000000000000000000000 P ff3e5dd931fb408ab774a9ac881109ae [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ff3e5dd931fb408ab774a9ac881109ae" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 40327 } }
I20260619 10:17:25.462241 5455 leader_election.cc:302] T 00000000000000000000000000000000 P ff3e5dd931fb408ab774a9ac881109ae [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ff3e5dd931fb408ab774a9ac881109ae; no voters:
I20260619 10:17:25.462411 5455 leader_election.cc:288] T 00000000000000000000000000000000 P ff3e5dd931fb408ab774a9ac881109ae [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 10:17:25.462461 5460 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ff3e5dd931fb408ab774a9ac881109ae [term 1 FOLLOWER]: Leader election won for term 1
I20260619 10:17:25.462633 5460 raft_consensus.cc:686] T 00000000000000000000000000000000 P ff3e5dd931fb408ab774a9ac881109ae [term 1 LEADER]: Becoming Leader. State: Replica: ff3e5dd931fb408ab774a9ac881109ae, State: Running, Role: LEADER
I20260619 10:17:25.462688 5455 sys_catalog.cc:564] T 00000000000000000000000000000000 P ff3e5dd931fb408ab774a9ac881109ae [sys.catalog]: configured and running, proceeding with master startup.
I20260619 10:17:25.462728 5460 consensus_queue.cc:237] T 00000000000000000000000000000000 P ff3e5dd931fb408ab774a9ac881109ae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ff3e5dd931fb408ab774a9ac881109ae" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 40327 } }
I20260619 10:17:25.463074 5462 sys_catalog.cc:455] T 00000000000000000000000000000000 P ff3e5dd931fb408ab774a9ac881109ae [sys.catalog]: SysCatalogTable state changed. Reason: New leader ff3e5dd931fb408ab774a9ac881109ae. Latest consensus state: current_term: 1 leader_uuid: "ff3e5dd931fb408ab774a9ac881109ae" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ff3e5dd931fb408ab774a9ac881109ae" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 40327 } } }
I20260619 10:17:25.463080 5461 sys_catalog.cc:455] T 00000000000000000000000000000000 P ff3e5dd931fb408ab774a9ac881109ae [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ff3e5dd931fb408ab774a9ac881109ae" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ff3e5dd931fb408ab774a9ac881109ae" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 40327 } } }
I20260619 10:17:25.463274 5461 sys_catalog.cc:458] T 00000000000000000000000000000000 P ff3e5dd931fb408ab774a9ac881109ae [sys.catalog]: This master's current role is: LEADER
I20260619 10:17:25.463225 5462 sys_catalog.cc:458] T 00000000000000000000000000000000 P ff3e5dd931fb408ab774a9ac881109ae [sys.catalog]: This master's current role is: LEADER
I20260619 10:17:25.463733 5469 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 10:17:25.464186 5469 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 10:17:25.465488 5469 catalog_manager.cc:1347] Generated new cluster ID: 746c8e082e8642ce80e2f7f5a579c330
I20260619 10:17:25.465543 5469 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 10:17:25.483377 5469 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 10:17:25.483551 5469 catalog_manager.cc:1504] Loading token signing keys...
I20260619 10:17:25.486392 5469 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ff3e5dd931fb408ab774a9ac881109ae: Generated new TSK 0
I20260619 10:17:25.486554 5469 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 10:17:25.495067 825 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTU9OP6/build/release/bin/kudu
/tmp/dist-test-taskTU9OP6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781864224577058-825-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781864224577058-825-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781864224577058-825-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781864224577058-825-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.206.65:0
--local_ip_for_outbound_sockets=127.0.206.65
--webserver_interface=127.0.206.65
--webserver_port=0
--tserver_master_addrs=127.0.206.126:40327
--builtin_ntp_servers=127.0.206.84:41839
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260619 10:17:25.574786 5479 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 10:17:25.575012 5479 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 10:17:25.575058 5479 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 10:17:25.576195 5479 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 10:17:25.576257 5479 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 10:17:25.576297 5479 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.206.65
I20260619 10:17:25.577626 5479 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.206.84:41839
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781864224577058-825-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781864224577058-825-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.206.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781864224577058-825-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.206.65
--webserver_port=0
--tserver_master_addrs=127.0.206.126:40327
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5479
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.206.65
--log_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781864224577058-825-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 09:32:44 UTC on ccycloud.quasar-gbtldp.root.comops.site
build id 4455440
I20260619 10:17:25.577876 5479 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 10:17:25.578159 5479 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 10:17:25.583838 5484 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 10:17:25.583838 5485 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 10:17:25.584003 5479 server_base.cc:1034] running on GCE node
W20260619 10:17:25.584110 5487 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 10:17:25.584342 5479 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 10:17:25.584604 5479 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 10:17:25.585806 5479 hybrid_clock.cc:648] HybridClock initialized: now 1781864245585800 us; error 40 us; skew 500 ppm
I20260619 10:17:25.587045 5479 webserver.cc:460] Webserver started at http://127.0.206.65:43291/ using document root <none> and password file <none>
I20260619 10:17:25.587299 5479 fs_manager.cc:359] Metadata directory not provided
I20260619 10:17:25.587343 5479 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 10:17:25.587502 5479 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 10:17:25.588563 5479 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781864224577058-825-0/minicluster-data/ts-0/data/instance:
uuid: "75531f2c748944968122e95dc7df2f56"
format_stamp: "Formatted at 2026-06-19 10:17:25 on dist-test-slave-6pt2"
server_key: "494bb9f3754a030578a47bfdd1fca20f"
server_key_iv: "0e0d6332877b2c98a57b4f211f9e5e51"
server_key_version: "encryptionkey@0"
I20260619 10:17:25.588913 5479 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781864224577058-825-0/minicluster-data/ts-0/wal/instance:
uuid: "75531f2c748944968122e95dc7df2f56"
format_stamp: "Formatted at 2026-06-19 10:17:25 on dist-test-slave-6pt2"
server_key: "494bb9f3754a030578a47bfdd1fca20f"
server_key_iv: "0e0d6332877b2c98a57b4f211f9e5e51"
server_key_version: "encryptionkey@0"
I20260619 10:17:25.590205 5479 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260619 10:17:25.590984 5493 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 10:17:25.591169 5479 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260619 10:17:25.591241 5479 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781864224577058-825-0/minicluster-data/ts-0/data,/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781864224577058-825-0/minicluster-data/ts-0/wal
uuid: "75531f2c748944968122e95dc7df2f56"
format_stamp: "Formatted at 2026-06-19 10:17:25 on dist-test-slave-6pt2"
server_key: "494bb9f3754a030578a47bfdd1fca20f"
server_key_iv: "0e0d6332877b2c98a57b4f211f9e5e51"
server_key_version: "encryptionkey@0"
I20260619 10:17:25.591331 5479 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781864224577058-825-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781864224577058-825-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781864224577058-825-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 10:17:25.594592 5479 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 10:17:25.594908 5479 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 10:17:25.595042 5479 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 10:17:25.595268 5479 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 10:17:25.595564 5479 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 10:17:25.595597 5479 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 10:17:25.595615 5479 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 10:17:25.595660 5479 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 10:17:25.601364 5479 rpc_server.cc:307] RPC server started. Bound to: 127.0.206.65:38719
I20260619 10:17:25.601402 5606 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.206.65:38719 every 8 connection(s)
I20260619 10:17:25.601717 5479 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781864224577058-825-0/minicluster-data/ts-0/data/info.pb
I20260619 10:17:25.605262 5458 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:25.602416 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.65:59677 (local address 127.0.206.126:40327)
0619 10:17:25.602496 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0619 10:17:25.602498 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 10:17:25.602754 (+ 256us) server_negotiation.cc:407] Connection header received
0619 10:17:25.603182 (+ 428us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:17:25.603184 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:17:25.603188 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:17:25.603241 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:17:25.604471 (+ 1230us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 10:17:25.604472 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 10:17:25.604472 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 10:17:25.604477 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0619 10:17:25.604490 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 10:17:25.604495 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 10:17:25.604560 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0619 10:17:25.605169 (+ 609us) server_negotiation.cc:299] Negotiation successful
0619 10:17:25.605186 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260619 10:17:25.605511 5609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:25.602385 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:40327 (local address 127.0.206.65:59677)
0619 10:17:25.602580 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0619 10:17:25.602653 (+ 73us) client_negotiation.cc:174] Beginning negotiation
0619 10:17:25.603102 (+ 449us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 10:17:25.603290 (+ 188us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 10:17:25.603292 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 10:17:25.603300 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0619 10:17:25.604298 (+ 998us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 10:17:25.604301 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0619 10:17:25.604340 (+ 39us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 10:17:25.604379 (+ 39us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 10:17:25.605066 (+ 687us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 10:17:25.605072 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 10:17:25.605075 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 10:17:25.605154 (+ 79us) client_negotiation.cc:240] Negotiation successful
0619 10:17:25.605231 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":99,"threads_started":1}
I20260619 10:17:25.606073 5607 heartbeater.cc:347] Connected to a master server at 127.0.206.126:40327
I20260619 10:17:25.606194 5607 heartbeater.cc:464] Registering TS with master...
I20260619 10:17:25.606345 5607 heartbeater.cc:511] Master 127.0.206.126:40327 requested a full tablet report, sending...
I20260619 10:17:25.606992 5419 ts_manager.cc:194] Registered new tserver with Master: 75531f2c748944968122e95dc7df2f56 (127.0.206.65:38719)
I20260619 10:17:25.609800 825 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTU9OP6/build/release/bin/kudu as pid 5479
I20260619 10:17:25.609872 825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781864224577058-825-0/minicluster-data/ts-0/wal/instance
I20260619 10:17:25.610064 825 external_mini_cluster.cc:1428] Setting key 636193d95f60292f528e51d7fbd68825
I20260619 10:17:25.610925 825 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTU9OP6/build/release/bin/kudu
/tmp/dist-test-taskTU9OP6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781864224577058-825-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781864224577058-825-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781864224577058-825-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781864224577058-825-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.206.66:0
--local_ip_for_outbound_sockets=127.0.206.66
--webserver_interface=127.0.206.66
--webserver_port=0
--tserver_master_addrs=127.0.206.126:40327
--builtin_ntp_servers=127.0.206.84:41839
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260619 10:17:25.690281 5610 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 10:17:25.690546 5610 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 10:17:25.690582 5610 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 10:17:25.691804 5610 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 10:17:25.691880 5610 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 10:17:25.691926 5610 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.206.66
I20260619 10:17:25.693318 5610 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.206.84:41839
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781864224577058-825-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781864224577058-825-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.206.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781864224577058-825-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.206.66
--webserver_port=0
--tserver_master_addrs=127.0.206.126:40327
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5610
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.206.66
--log_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781864224577058-825-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 09:32:44 UTC on ccycloud.quasar-gbtldp.root.comops.site
build id 4455440
I20260619 10:17:25.693602 5610 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 10:17:25.693903 5610 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 10:17:25.698846 5618 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 10:17:25.698854 5616 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 10:17:25.698854 5615 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 10:17:25.698870 5610 server_base.cc:1034] running on GCE node
I20260619 10:17:25.699361 5610 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 10:17:25.699613 5610 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 10:17:25.700775 5610 hybrid_clock.cc:648] HybridClock initialized: now 1781864245700768 us; error 39 us; skew 500 ppm
I20260619 10:17:25.702003 5610 webserver.cc:460] Webserver started at http://127.0.206.66:34477/ using document root <none> and password file <none>
I20260619 10:17:25.702252 5610 fs_manager.cc:359] Metadata directory not provided
I20260619 10:17:25.702322 5610 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 10:17:25.702461 5610 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 10:17:25.703636 5610 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781864224577058-825-0/minicluster-data/ts-1/data/instance:
uuid: "ddd9f3878b1a46e0a8cb9b64cd38f40c"
format_stamp: "Formatted at 2026-06-19 10:17:25 on dist-test-slave-6pt2"
server_key: "2ed1074c598e3c0d93a23860e44b8477"
server_key_iv: "1898e6e66569f4f29f7321a6b95e788f"
server_key_version: "encryptionkey@0"
I20260619 10:17:25.704011 5610 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781864224577058-825-0/minicluster-data/ts-1/wal/instance:
uuid: "ddd9f3878b1a46e0a8cb9b64cd38f40c"
format_stamp: "Formatted at 2026-06-19 10:17:25 on dist-test-slave-6pt2"
server_key: "2ed1074c598e3c0d93a23860e44b8477"
server_key_iv: "1898e6e66569f4f29f7321a6b95e788f"
server_key_version: "encryptionkey@0"
I20260619 10:17:25.705245 5610 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 10:17:25.705920 5624 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 10:17:25.706094 5610 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 10:17:25.706192 5610 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781864224577058-825-0/minicluster-data/ts-1/data,/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781864224577058-825-0/minicluster-data/ts-1/wal
uuid: "ddd9f3878b1a46e0a8cb9b64cd38f40c"
format_stamp: "Formatted at 2026-06-19 10:17:25 on dist-test-slave-6pt2"
server_key: "2ed1074c598e3c0d93a23860e44b8477"
server_key_iv: "1898e6e66569f4f29f7321a6b95e788f"
server_key_version: "encryptionkey@0"
I20260619 10:17:25.706269 5610 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781864224577058-825-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781864224577058-825-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781864224577058-825-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 10:17:25.709553 5610 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 10:17:25.709810 5610 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 10:17:25.709950 5610 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 10:17:25.710158 5610 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 10:17:25.710464 5610 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 10:17:25.710515 5610 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 10:17:25.710553 5610 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 10:17:25.710579 5610 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 10:17:25.716184 5610 rpc_server.cc:307] RPC server started. Bound to: 127.0.206.66:39203
I20260619 10:17:25.716223 5737 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.206.66:39203 every 8 connection(s)
I20260619 10:17:25.716532 5610 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781864224577058-825-0/minicluster-data/ts-1/data/info.pb
I20260619 10:17:25.719338 5458 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:25.717219 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.66:46741 (local address 127.0.206.126:40327)
0619 10:17:25.717305 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0619 10:17:25.717307 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 10:17:25.717502 (+ 195us) server_negotiation.cc:407] Connection header received
0619 10:17:25.717932 (+ 430us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:17:25.717933 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:17:25.717938 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:17:25.718008 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:17:25.719172 (+ 1164us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 10:17:25.719172 (+ 0us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 10:17:25.719173 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 10:17:25.719176 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0619 10:17:25.719187 (+ 11us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 10:17:25.719190 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 10:17:25.719232 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0619 10:17:25.719277 (+ 45us) server_negotiation.cc:299] Negotiation successful
0619 10:17:25.719289 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260619 10:17:25.719574 5740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:25.717185 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:40327 (local address 127.0.206.66:46741)
0619 10:17:25.717385 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0619 10:17:25.717415 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0619 10:17:25.717856 (+ 441us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 10:17:25.718058 (+ 202us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 10:17:25.718061 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 10:17:25.718070 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0619 10:17:25.719046 (+ 976us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 10:17:25.719047 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0619 10:17:25.719073 (+ 26us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 10:17:25.719099 (+ 26us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 10:17:25.719231 (+ 132us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 10:17:25.719233 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 10:17:25.719235 (+ 2us) client_negotiation.cc:769] Sending connection context
0619 10:17:25.719285 (+ 50us) client_negotiation.cc:240] Negotiation successful
0619 10:17:25.719330 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":95,"threads_started":1}
I20260619 10:17:25.720078 5738 heartbeater.cc:347] Connected to a master server at 127.0.206.126:40327
I20260619 10:17:25.720171 5738 heartbeater.cc:464] Registering TS with master...
I20260619 10:17:25.720315 5738 heartbeater.cc:511] Master 127.0.206.126:40327 requested a full tablet report, sending...
I20260619 10:17:25.720678 5419 ts_manager.cc:194] Registered new tserver with Master: ddd9f3878b1a46e0a8cb9b64cd38f40c (127.0.206.66:39203)
I20260619 10:17:25.725111 825 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTU9OP6/build/release/bin/kudu as pid 5610
I20260619 10:17:25.725186 825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781864224577058-825-0/minicluster-data/ts-1/wal/instance
I20260619 10:17:25.725392 825 external_mini_cluster.cc:1428] Setting key 04fb2d6673a41627b988124ace61ae5d
I20260619 10:17:25.726244 825 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTU9OP6/build/release/bin/kudu
/tmp/dist-test-taskTU9OP6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781864224577058-825-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781864224577058-825-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781864224577058-825-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781864224577058-825-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.206.67:0
--local_ip_for_outbound_sockets=127.0.206.67
--webserver_interface=127.0.206.67
--webserver_port=0
--tserver_master_addrs=127.0.206.126:40327
--builtin_ntp_servers=127.0.206.84:41839
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260619 10:17:25.804999 5741 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 10:17:25.805189 5741 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 10:17:25.805208 5741 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 10:17:25.806316 5741 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 10:17:25.806353 5741 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 10:17:25.806377 5741 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.206.67
I20260619 10:17:25.807659 5741 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.206.84:41839
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781864224577058-825-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781864224577058-825-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.206.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781864224577058-825-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.206.67
--webserver_port=0
--tserver_master_addrs=127.0.206.126:40327
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5741
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.206.67
--log_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781864224577058-825-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 09:32:44 UTC on ccycloud.quasar-gbtldp.root.comops.site
build id 4455440
I20260619 10:17:25.807881 5741 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 10:17:25.808173 5741 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 10:17:25.812945 5747 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 10:17:25.812942 5749 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 10:17:25.813072 5746 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 10:17:25.813221 5741 server_base.cc:1034] running on GCE node
I20260619 10:17:25.813441 5741 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 10:17:25.813669 5741 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 10:17:25.814823 5741 hybrid_clock.cc:648] HybridClock initialized: now 1781864245814818 us; error 35 us; skew 500 ppm
I20260619 10:17:25.815948 5741 webserver.cc:460] Webserver started at http://127.0.206.67:46061/ using document root <none> and password file <none>
I20260619 10:17:25.816185 5741 fs_manager.cc:359] Metadata directory not provided
I20260619 10:17:25.816273 5741 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 10:17:25.816431 5741 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 10:17:25.817452 5741 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781864224577058-825-0/minicluster-data/ts-2/data/instance:
uuid: "8fba367aecce477ebf2490ce10206f90"
format_stamp: "Formatted at 2026-06-19 10:17:25 on dist-test-slave-6pt2"
server_key: "93104416902df9d214c3c6d1eacb9612"
server_key_iv: "9895fb815cb1c61791f18462adee1f5b"
server_key_version: "encryptionkey@0"
I20260619 10:17:25.817792 5741 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781864224577058-825-0/minicluster-data/ts-2/wal/instance:
uuid: "8fba367aecce477ebf2490ce10206f90"
format_stamp: "Formatted at 2026-06-19 10:17:25 on dist-test-slave-6pt2"
server_key: "93104416902df9d214c3c6d1eacb9612"
server_key_iv: "9895fb815cb1c61791f18462adee1f5b"
server_key_version: "encryptionkey@0"
I20260619 10:17:25.819057 5741 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 10:17:25.819700 5755 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 10:17:25.819850 5741 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 10:17:25.819944 5741 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781864224577058-825-0/minicluster-data/ts-2/data,/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781864224577058-825-0/minicluster-data/ts-2/wal
uuid: "8fba367aecce477ebf2490ce10206f90"
format_stamp: "Formatted at 2026-06-19 10:17:25 on dist-test-slave-6pt2"
server_key: "93104416902df9d214c3c6d1eacb9612"
server_key_iv: "9895fb815cb1c61791f18462adee1f5b"
server_key_version: "encryptionkey@0"
I20260619 10:17:25.820020 5741 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781864224577058-825-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781864224577058-825-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781864224577058-825-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 10:17:25.823361 5741 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 10:17:25.823608 5741 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 10:17:25.823742 5741 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 10:17:25.823959 5741 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 10:17:25.824326 5741 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 10:17:25.824379 5741 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 10:17:25.824409 5741 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 10:17:25.824441 5741 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 10:17:25.829953 5741 rpc_server.cc:307] RPC server started. Bound to: 127.0.206.67:42943
I20260619 10:17:25.830070 5868 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.206.67:42943 every 8 connection(s)
I20260619 10:17:25.830338 5741 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781864224577058-825-0/minicluster-data/ts-2/data/info.pb
I20260619 10:17:25.830600 825 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTU9OP6/build/release/bin/kudu as pid 5741
I20260619 10:17:25.830715 825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781864224577058-825-0/minicluster-data/ts-2/wal/instance
I20260619 10:17:25.830901 825 external_mini_cluster.cc:1428] Setting key b93a6e3cba07d3f83ee9ecfbc0e1bc38
I20260619 10:17:25.833300 5458 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:25.831144 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.67:35783 (local address 127.0.206.126:40327)
0619 10:17:25.831191 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0619 10:17:25.831193 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 10:17:25.831482 (+ 289us) server_negotiation.cc:407] Connection header received
0619 10:17:25.832050 (+ 568us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:17:25.832051 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:17:25.832055 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:17:25.832109 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:17:25.833127 (+ 1018us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 10:17:25.833128 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 10:17:25.833129 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 10:17:25.833132 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0619 10:17:25.833143 (+ 11us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 10:17:25.833147 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 10:17:25.833190 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0619 10:17:25.833238 (+ 48us) server_negotiation.cc:299] Negotiation successful
0619 10:17:25.833250 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260619 10:17:25.833501 5871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:25.831142 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:40327 (local address 127.0.206.67:35783)
0619 10:17:25.831367 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0619 10:17:25.831406 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0619 10:17:25.831955 (+ 549us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 10:17:25.832156 (+ 201us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 10:17:25.832159 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 10:17:25.832166 (+ 7us) client_negotiation.cc:189] Negotiated authn=SASL
0619 10:17:25.833001 (+ 835us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 10:17:25.833002 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0619 10:17:25.833027 (+ 25us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 10:17:25.833052 (+ 25us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 10:17:25.833189 (+ 137us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 10:17:25.833191 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 10:17:25.833193 (+ 2us) client_negotiation.cc:769] Sending connection context
0619 10:17:25.833237 (+ 44us) client_negotiation.cc:240] Negotiation successful
0619 10:17:25.833277 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":110,"threads_started":1}
I20260619 10:17:25.833910 5869 heartbeater.cc:347] Connected to a master server at 127.0.206.126:40327
I20260619 10:17:25.833981 5869 heartbeater.cc:464] Registering TS with master...
I20260619 10:17:25.834136 5869 heartbeater.cc:511] Master 127.0.206.126:40327 requested a full tablet report, sending...
I20260619 10:17:25.834447 5419 ts_manager.cc:194] Registered new tserver with Master: 8fba367aecce477ebf2490ce10206f90 (127.0.206.67:42943)
I20260619 10:17:25.842461 825 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 10:17:25.844751 5458 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:25.843717 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36156 (local address 127.0.206.126:40327)
0619 10:17:25.843783 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0619 10:17:25.843785 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 10:17:25.843875 (+ 90us) server_negotiation.cc:407] Connection header received
0619 10:17:25.843956 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:17:25.843957 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:17:25.843962 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:17:25.844015 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:17:25.844538 (+ 523us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 10:17:25.844540 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 10:17:25.844540 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 10:17:25.844544 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0619 10:17:25.844556 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 10:17:25.844560 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 10:17:25.844616 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0619 10:17:25.844670 (+ 54us) server_negotiation.cc:299] Negotiation successful
0619 10:17:25.844684 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260619 10:17:25.846182 5419 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:36156:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 10:17:25.847736 5419 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 10:17:25.851486 5878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:25.849840 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.65:38719 (local address 127.0.0.1:43996)
0619 10:17:25.850005 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0619 10:17:25.850041 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0619 10:17:25.850095 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 10:17:25.850554 (+ 459us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 10:17:25.850555 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 10:17:25.850557 (+ 2us) client_negotiation.cc:189] Negotiated authn=SASL
0619 10:17:25.851036 (+ 479us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 10:17:25.851036 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0619 10:17:25.851041 (+ 5us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 10:17:25.851049 (+ 8us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 10:17:25.851331 (+ 282us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 10:17:25.851336 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 10:17:25.851339 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 10:17:25.851409 (+ 70us) client_negotiation.cc:240] Negotiation successful
0619 10:17:25.851422 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":59,"threads_started":1}
I20260619 10:17:25.851552 5882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:25.850217 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43996 (local address 127.0.206.65:38719)
0619 10:17:25.850383 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0619 10:17:25.850387 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 10:17:25.850396 (+ 9us) server_negotiation.cc:407] Connection header received
0619 10:17:25.850420 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:17:25.850422 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:17:25.850430 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:17:25.850764 (+ 334us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:17:25.851136 (+ 372us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 10:17:25.851139 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 10:17:25.851140 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 10:17:25.851143 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0619 10:17:25.851171 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 10:17:25.851238 (+ 67us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 10:17:25.851335 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0619 10:17:25.851446 (+ 111us) server_negotiation.cc:299] Negotiation successful
0619 10:17:25.851462 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":113,"thread_start_us":76,"threads_started":1}
I20260619 10:17:25.851606 5879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:25.849855 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.67:42943 (local address 127.0.0.1:41792)
0619 10:17:25.850384 (+ 529us) negotiation.cc:107] Waiting for socket to connect
0619 10:17:25.850394 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 10:17:25.850437 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 10:17:25.850659 (+ 222us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 10:17:25.850660 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 10:17:25.850663 (+ 3us) client_negotiation.cc:189] Negotiated authn=SASL
0619 10:17:25.851216 (+ 553us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 10:17:25.851217 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0619 10:17:25.851224 (+ 7us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 10:17:25.851235 (+ 11us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 10:17:25.851467 (+ 232us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 10:17:25.851468 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 10:17:25.851469 (+ 1us) client_negotiation.cc:769] Sending connection context
0619 10:17:25.851525 (+ 56us) client_negotiation.cc:240] Negotiation successful
0619 10:17:25.851538 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":513,"thread_start_us":75,"threads_started":1}
I20260619 10:17:25.851864 5881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:25.850050 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41792 (local address 127.0.206.67:42943)
0619 10:17:25.850228 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0619 10:17:25.850233 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 10:17:25.850501 (+ 268us) server_negotiation.cc:407] Connection header received
0619 10:17:25.850526 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:17:25.850529 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:17:25.850538 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:17:25.850798 (+ 260us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:17:25.851325 (+ 527us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 10:17:25.851332 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 10:17:25.851332 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 10:17:25.851337 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0619 10:17:25.851369 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 10:17:25.851391 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 10:17:25.851442 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0619 10:17:25.851765 (+ 323us) server_negotiation.cc:299] Negotiation successful
0619 10:17:25.851782 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":147,"thread_start_us":85,"threads_started":1}
I20260619 10:17:25.852098 5877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:25.849851 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.66:39203 (local address 127.0.0.1:41066)
0619 10:17:25.850179 (+ 328us) negotiation.cc:107] Waiting for socket to connect
0619 10:17:25.850189 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 10:17:25.850235 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 10:17:25.850397 (+ 162us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 10:17:25.850403 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 10:17:25.850410 (+ 7us) client_negotiation.cc:189] Negotiated authn=SASL
0619 10:17:25.850876 (+ 466us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 10:17:25.850877 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0619 10:17:25.850891 (+ 14us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 10:17:25.850912 (+ 21us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 10:17:25.851882 (+ 970us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 10:17:25.851883 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 10:17:25.851884 (+ 1us) client_negotiation.cc:769] Sending connection context
0619 10:17:25.852020 (+ 136us) client_negotiation.cc:240] Negotiation successful
0619 10:17:25.852033 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":316,"thread_start_us":69,"threads_started":1}
I20260619 10:17:25.852157 5880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:25.850010 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41066 (local address 127.0.206.66:39203)
0619 10:17:25.850188 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0619 10:17:25.850193 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 10:17:25.850231 (+ 38us) server_negotiation.cc:407] Connection header received
0619 10:17:25.850285 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:17:25.850288 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:17:25.850299 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:17:25.850383 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:17:25.851768 (+ 1385us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 10:17:25.851774 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 10:17:25.851775 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 10:17:25.851778 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0619 10:17:25.851809 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 10:17:25.851825 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 10:17:25.851888 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0619 10:17:25.852040 (+ 152us) server_negotiation.cc:299] Negotiation successful
0619 10:17:25.852057 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":143,"thread_start_us":116,"threads_started":1}
I20260619 10:17:25.852389 5541 tablet_service.cc:1467] Processing CreateTablet for tablet b8e7521ddff647c291fdda92596c0ef9 (DEFAULT_TABLE table=test-table [id=ee669a78623d428db4d2f516bd9efe5d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 10:17:25.852727 5541 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b8e7521ddff647c291fdda92596c0ef9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 10:17:25.852901 5672 tablet_service.cc:1467] Processing CreateTablet for tablet b8e7521ddff647c291fdda92596c0ef9 (DEFAULT_TABLE table=test-table [id=ee669a78623d428db4d2f516bd9efe5d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 10:17:25.853222 5672 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b8e7521ddff647c291fdda92596c0ef9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 10:17:25.855024 5803 tablet_service.cc:1467] Processing CreateTablet for tablet b8e7521ddff647c291fdda92596c0ef9 (DEFAULT_TABLE table=test-table [id=ee669a78623d428db4d2f516bd9efe5d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 10:17:25.855278 5883 tablet_bootstrap.cc:492] T b8e7521ddff647c291fdda92596c0ef9 P 75531f2c748944968122e95dc7df2f56: Bootstrap starting.
I20260619 10:17:25.855336 5803 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b8e7521ddff647c291fdda92596c0ef9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 10:17:25.855917 5884 tablet_bootstrap.cc:492] T b8e7521ddff647c291fdda92596c0ef9 P ddd9f3878b1a46e0a8cb9b64cd38f40c: Bootstrap starting.
I20260619 10:17:25.856215 5883 tablet_bootstrap.cc:654] T b8e7521ddff647c291fdda92596c0ef9 P 75531f2c748944968122e95dc7df2f56: Neither blocks nor log segments found. Creating new log.
I20260619 10:17:25.856534 5884 tablet_bootstrap.cc:654] T b8e7521ddff647c291fdda92596c0ef9 P ddd9f3878b1a46e0a8cb9b64cd38f40c: Neither blocks nor log segments found. Creating new log.
I20260619 10:17:25.856536 5883 log.cc:824] T b8e7521ddff647c291fdda92596c0ef9 P 75531f2c748944968122e95dc7df2f56: Log is configured to *not* fsync() on all Append() calls
I20260619 10:17:25.856830 5884 log.cc:824] T b8e7521ddff647c291fdda92596c0ef9 P ddd9f3878b1a46e0a8cb9b64cd38f40c: Log is configured to *not* fsync() on all Append() calls
I20260619 10:17:25.857300 5883 tablet_bootstrap.cc:492] T b8e7521ddff647c291fdda92596c0ef9 P 75531f2c748944968122e95dc7df2f56: No bootstrap required, opened a new log
I20260619 10:17:25.857368 5883 ts_tablet_manager.cc:1397] T b8e7521ddff647c291fdda92596c0ef9 P 75531f2c748944968122e95dc7df2f56: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260619 10:17:25.857437 5884 tablet_bootstrap.cc:492] T b8e7521ddff647c291fdda92596c0ef9 P ddd9f3878b1a46e0a8cb9b64cd38f40c: No bootstrap required, opened a new log
I20260619 10:17:25.857496 5884 ts_tablet_manager.cc:1397] T b8e7521ddff647c291fdda92596c0ef9 P ddd9f3878b1a46e0a8cb9b64cd38f40c: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260619 10:17:25.858196 5887 tablet_bootstrap.cc:492] T b8e7521ddff647c291fdda92596c0ef9 P 8fba367aecce477ebf2490ce10206f90: Bootstrap starting.
I20260619 10:17:25.858811 5887 tablet_bootstrap.cc:654] T b8e7521ddff647c291fdda92596c0ef9 P 8fba367aecce477ebf2490ce10206f90: Neither blocks nor log segments found. Creating new log.
I20260619 10:17:25.859124 5887 log.cc:824] T b8e7521ddff647c291fdda92596c0ef9 P 8fba367aecce477ebf2490ce10206f90: Log is configured to *not* fsync() on all Append() calls
I20260619 10:17:25.859910 5887 tablet_bootstrap.cc:492] T b8e7521ddff647c291fdda92596c0ef9 P 8fba367aecce477ebf2490ce10206f90: No bootstrap required, opened a new log
I20260619 10:17:25.859977 5887 ts_tablet_manager.cc:1397] T b8e7521ddff647c291fdda92596c0ef9 P 8fba367aecce477ebf2490ce10206f90: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260619 10:17:25.860247 5884 raft_consensus.cc:348] T b8e7521ddff647c291fdda92596c0ef9 P ddd9f3878b1a46e0a8cb9b64cd38f40c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ddd9f3878b1a46e0a8cb9b64cd38f40c" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 39203 } } peers { permanent_uuid: "75531f2c748944968122e95dc7df2f56" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 38719 } } peers { permanent_uuid: "8fba367aecce477ebf2490ce10206f90" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 42943 } }
I20260619 10:17:25.860407 5884 raft_consensus.cc:374] T b8e7521ddff647c291fdda92596c0ef9 P ddd9f3878b1a46e0a8cb9b64cd38f40c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 10:17:25.860441 5884 raft_consensus.cc:729] T b8e7521ddff647c291fdda92596c0ef9 P ddd9f3878b1a46e0a8cb9b64cd38f40c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ddd9f3878b1a46e0a8cb9b64cd38f40c, State: Initialized, Role: FOLLOWER
I20260619 10:17:25.860575 5884 consensus_queue.cc:260] T b8e7521ddff647c291fdda92596c0ef9 P ddd9f3878b1a46e0a8cb9b64cd38f40c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ddd9f3878b1a46e0a8cb9b64cd38f40c" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 39203 } } peers { permanent_uuid: "75531f2c748944968122e95dc7df2f56" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 38719 } } peers { permanent_uuid: "8fba367aecce477ebf2490ce10206f90" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 42943 } }
I20260619 10:17:25.860810 5884 ts_tablet_manager.cc:1428] T b8e7521ddff647c291fdda92596c0ef9 P ddd9f3878b1a46e0a8cb9b64cd38f40c: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260619 10:17:25.860927 5738 heartbeater.cc:503] Master 127.0.206.126:40327 was elected leader, sending a full tablet report...
I20260619 10:17:25.860868 5883 raft_consensus.cc:348] T b8e7521ddff647c291fdda92596c0ef9 P 75531f2c748944968122e95dc7df2f56 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ddd9f3878b1a46e0a8cb9b64cd38f40c" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 39203 } } peers { permanent_uuid: "75531f2c748944968122e95dc7df2f56" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 38719 } } peers { permanent_uuid: "8fba367aecce477ebf2490ce10206f90" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 42943 } }
I20260619 10:17:25.861022 5883 raft_consensus.cc:374] T b8e7521ddff647c291fdda92596c0ef9 P 75531f2c748944968122e95dc7df2f56 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 10:17:25.861066 5883 raft_consensus.cc:729] T b8e7521ddff647c291fdda92596c0ef9 P 75531f2c748944968122e95dc7df2f56 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 75531f2c748944968122e95dc7df2f56, State: Initialized, Role: FOLLOWER
I20260619 10:17:25.861186 5883 consensus_queue.cc:260] T b8e7521ddff647c291fdda92596c0ef9 P 75531f2c748944968122e95dc7df2f56 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ddd9f3878b1a46e0a8cb9b64cd38f40c" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 39203 } } peers { permanent_uuid: "75531f2c748944968122e95dc7df2f56" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 38719 } } peers { permanent_uuid: "8fba367aecce477ebf2490ce10206f90" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 42943 } }
I20260619 10:17:25.861419 5883 ts_tablet_manager.cc:1428] T b8e7521ddff647c291fdda92596c0ef9 P 75531f2c748944968122e95dc7df2f56: Time spent starting tablet: real 0.004s user 0.003s sys 0.000s
I20260619 10:17:25.861442 5607 heartbeater.cc:503] Master 127.0.206.126:40327 was elected leader, sending a full tablet report...
I20260619 10:17:25.862493 5887 raft_consensus.cc:348] T b8e7521ddff647c291fdda92596c0ef9 P 8fba367aecce477ebf2490ce10206f90 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ddd9f3878b1a46e0a8cb9b64cd38f40c" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 39203 } } peers { permanent_uuid: "75531f2c748944968122e95dc7df2f56" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 38719 } } peers { permanent_uuid: "8fba367aecce477ebf2490ce10206f90" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 42943 } }
I20260619 10:17:25.862650 5887 raft_consensus.cc:374] T b8e7521ddff647c291fdda92596c0ef9 P 8fba367aecce477ebf2490ce10206f90 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 10:17:25.862689 5887 raft_consensus.cc:729] T b8e7521ddff647c291fdda92596c0ef9 P 8fba367aecce477ebf2490ce10206f90 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8fba367aecce477ebf2490ce10206f90, State: Initialized, Role: FOLLOWER
I20260619 10:17:25.862771 5887 consensus_queue.cc:260] T b8e7521ddff647c291fdda92596c0ef9 P 8fba367aecce477ebf2490ce10206f90 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ddd9f3878b1a46e0a8cb9b64cd38f40c" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 39203 } } peers { permanent_uuid: "75531f2c748944968122e95dc7df2f56" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 38719 } } peers { permanent_uuid: "8fba367aecce477ebf2490ce10206f90" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 42943 } }
I20260619 10:17:25.862995 5887 ts_tablet_manager.cc:1428] T b8e7521ddff647c291fdda92596c0ef9 P 8fba367aecce477ebf2490ce10206f90: Time spent starting tablet: real 0.003s user 0.001s sys 0.002s
I20260619 10:17:25.863023 5869 heartbeater.cc:503] Master 127.0.206.126:40327 was elected leader, sending a full tablet report...
W20260619 10:17:25.967039 5739 tablet.cc:2369] T b8e7521ddff647c291fdda92596c0ef9 P ddd9f3878b1a46e0a8cb9b64cd38f40c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 10:17:25.994545 5891 raft_consensus.cc:482] T b8e7521ddff647c291fdda92596c0ef9 P 8fba367aecce477ebf2490ce10206f90 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 10:17:25.994681 5891 raft_consensus.cc:504] T b8e7521ddff647c291fdda92596c0ef9 P 8fba367aecce477ebf2490ce10206f90 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ddd9f3878b1a46e0a8cb9b64cd38f40c" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 39203 } } peers { permanent_uuid: "75531f2c748944968122e95dc7df2f56" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 38719 } } peers { permanent_uuid: "8fba367aecce477ebf2490ce10206f90" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 42943 } }
I20260619 10:17:25.995080 5891 leader_election.cc:288] T b8e7521ddff647c291fdda92596c0ef9 P 8fba367aecce477ebf2490ce10206f90 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ddd9f3878b1a46e0a8cb9b64cd38f40c (127.0.206.66:39203), 75531f2c748944968122e95dc7df2f56 (127.0.206.65:38719)
I20260619 10:17:25.996402 5871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:25.995248 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.65:38719 (local address 127.0.206.67:37777)
0619 10:17:25.995300 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0619 10:17:25.995311 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 10:17:25.995348 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 10:17:25.995517 (+ 169us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 10:17:25.995518 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 10:17:25.995521 (+ 3us) client_negotiation.cc:189] Negotiated authn=SASL
0619 10:17:25.996065 (+ 544us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 10:17:25.996066 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0619 10:17:25.996072 (+ 6us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 10:17:25.996083 (+ 11us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 10:17:25.996257 (+ 174us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 10:17:25.996258 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 10:17:25.996259 (+ 1us) client_negotiation.cc:769] Sending connection context
0619 10:17:25.996308 (+ 49us) client_negotiation.cc:240] Negotiation successful
0619 10:17:25.996323 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33}
I20260619 10:17:25.996402 5882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:25.995316 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.67:37777 (local address 127.0.206.65:38719)
0619 10:17:25.995380 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0619 10:17:25.995383 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 10:17:25.995398 (+ 15us) server_negotiation.cc:407] Connection header received
0619 10:17:25.995425 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:17:25.995426 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:17:25.995438 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:17:25.995487 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:17:25.996164 (+ 677us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 10:17:25.996165 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 10:17:25.996165 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 10:17:25.996170 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0619 10:17:25.996189 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 10:17:25.996198 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 10:17:25.996262 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0619 10:17:25.996309 (+ 47us) server_negotiation.cc:299] Negotiation successful
0619 10:17:25.996323 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260619 10:17:25.996587 5880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:25.995319 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.67:47547 (local address 127.0.206.66:39203)
0619 10:17:25.995389 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0619 10:17:25.995391 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 10:17:25.995669 (+ 278us) server_negotiation.cc:407] Connection header received
0619 10:17:25.995710 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:17:25.995711 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:17:25.995723 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:17:25.995766 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:17:25.996350 (+ 584us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 10:17:25.996351 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 10:17:25.996352 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 10:17:25.996357 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0619 10:17:25.996384 (+ 27us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 10:17:25.996388 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 10:17:25.996443 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0619 10:17:25.996508 (+ 65us) server_negotiation.cc:299] Negotiation successful
0619 10:17:25.996522 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260619 10:17:25.996625 5892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:25.995249 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.66:39203 (local address 127.0.206.67:47547)
0619 10:17:25.995596 (+ 347us) negotiation.cc:107] Waiting for socket to connect
0619 10:17:25.995604 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0619 10:17:25.995647 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 10:17:25.995788 (+ 141us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 10:17:25.995789 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 10:17:25.995792 (+ 3us) client_negotiation.cc:189] Negotiated authn=SASL
0619 10:17:25.996236 (+ 444us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 10:17:25.996236 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0619 10:17:25.996243 (+ 7us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 10:17:25.996269 (+ 26us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 10:17:25.996444 (+ 175us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 10:17:25.996445 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 10:17:25.996446 (+ 1us) client_negotiation.cc:769] Sending connection context
0619 10:17:25.996510 (+ 64us) client_negotiation.cc:240] Negotiation successful
0619 10:17:25.996522 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":327,"mutex_wait_us":169,"thread_start_us":101,"threads_started":1}
I20260619 10:17:25.997067 5692 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b8e7521ddff647c291fdda92596c0ef9" candidate_uuid: "8fba367aecce477ebf2490ce10206f90" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ddd9f3878b1a46e0a8cb9b64cd38f40c" is_pre_election: true
I20260619 10:17:25.997118 5561 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b8e7521ddff647c291fdda92596c0ef9" candidate_uuid: "8fba367aecce477ebf2490ce10206f90" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75531f2c748944968122e95dc7df2f56" is_pre_election: true
I20260619 10:17:25.997225 5692 raft_consensus.cc:2454] T b8e7521ddff647c291fdda92596c0ef9 P ddd9f3878b1a46e0a8cb9b64cd38f40c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fba367aecce477ebf2490ce10206f90 in term 0.
I20260619 10:17:25.997233 5561 raft_consensus.cc:2454] T b8e7521ddff647c291fdda92596c0ef9 P 75531f2c748944968122e95dc7df2f56 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fba367aecce477ebf2490ce10206f90 in term 0.
I20260619 10:17:25.997447 5757 leader_election.cc:302] T b8e7521ddff647c291fdda92596c0ef9 P 8fba367aecce477ebf2490ce10206f90 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fba367aecce477ebf2490ce10206f90, ddd9f3878b1a46e0a8cb9b64cd38f40c; no voters:
I20260619 10:17:25.997622 5891 raft_consensus.cc:2781] T b8e7521ddff647c291fdda92596c0ef9 P 8fba367aecce477ebf2490ce10206f90 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 10:17:25.997661 5891 raft_consensus.cc:482] T b8e7521ddff647c291fdda92596c0ef9 P 8fba367aecce477ebf2490ce10206f90 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 10:17:25.997676 5891 raft_consensus.cc:3037] T b8e7521ddff647c291fdda92596c0ef9 P 8fba367aecce477ebf2490ce10206f90 [term 0 FOLLOWER]: Advancing to term 1
I20260619 10:17:25.998332 5891 raft_consensus.cc:504] T b8e7521ddff647c291fdda92596c0ef9 P 8fba367aecce477ebf2490ce10206f90 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ddd9f3878b1a46e0a8cb9b64cd38f40c" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 39203 } } peers { permanent_uuid: "75531f2c748944968122e95dc7df2f56" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 38719 } } peers { permanent_uuid: "8fba367aecce477ebf2490ce10206f90" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 42943 } }
I20260619 10:17:25.998463 5891 leader_election.cc:288] T b8e7521ddff647c291fdda92596c0ef9 P 8fba367aecce477ebf2490ce10206f90 [CANDIDATE]: Term 1 election: Requested vote from peers ddd9f3878b1a46e0a8cb9b64cd38f40c (127.0.206.66:39203), 75531f2c748944968122e95dc7df2f56 (127.0.206.65:38719)
I20260619 10:17:25.998646 5692 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b8e7521ddff647c291fdda92596c0ef9" candidate_uuid: "8fba367aecce477ebf2490ce10206f90" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ddd9f3878b1a46e0a8cb9b64cd38f40c"
I20260619 10:17:25.998667 5561 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b8e7521ddff647c291fdda92596c0ef9" candidate_uuid: "8fba367aecce477ebf2490ce10206f90" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75531f2c748944968122e95dc7df2f56"
I20260619 10:17:25.998723 5692 raft_consensus.cc:3037] T b8e7521ddff647c291fdda92596c0ef9 P ddd9f3878b1a46e0a8cb9b64cd38f40c [term 0 FOLLOWER]: Advancing to term 1
I20260619 10:17:25.998731 5561 raft_consensus.cc:3037] T b8e7521ddff647c291fdda92596c0ef9 P 75531f2c748944968122e95dc7df2f56 [term 0 FOLLOWER]: Advancing to term 1
I20260619 10:17:25.999624 5561 raft_consensus.cc:2454] T b8e7521ddff647c291fdda92596c0ef9 P 75531f2c748944968122e95dc7df2f56 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fba367aecce477ebf2490ce10206f90 in term 1.
I20260619 10:17:25.999627 5692 raft_consensus.cc:2454] T b8e7521ddff647c291fdda92596c0ef9 P ddd9f3878b1a46e0a8cb9b64cd38f40c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fba367aecce477ebf2490ce10206f90 in term 1.
I20260619 10:17:25.999804 5756 leader_election.cc:302] T b8e7521ddff647c291fdda92596c0ef9 P 8fba367aecce477ebf2490ce10206f90 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 75531f2c748944968122e95dc7df2f56, 8fba367aecce477ebf2490ce10206f90; no voters:
I20260619 10:17:25.999903 5891 raft_consensus.cc:2781] T b8e7521ddff647c291fdda92596c0ef9 P 8fba367aecce477ebf2490ce10206f90 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 10:17:26.000063 5891 raft_consensus.cc:686] T b8e7521ddff647c291fdda92596c0ef9 P 8fba367aecce477ebf2490ce10206f90 [term 1 LEADER]: Becoming Leader. State: Replica: 8fba367aecce477ebf2490ce10206f90, State: Running, Role: LEADER
I20260619 10:17:26.000164 5891 consensus_queue.cc:237] T b8e7521ddff647c291fdda92596c0ef9 P 8fba367aecce477ebf2490ce10206f90 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ddd9f3878b1a46e0a8cb9b64cd38f40c" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 39203 } } peers { permanent_uuid: "75531f2c748944968122e95dc7df2f56" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 38719 } } peers { permanent_uuid: "8fba367aecce477ebf2490ce10206f90" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 42943 } }
I20260619 10:17:26.000931 5419 catalog_manager.cc:5495] T b8e7521ddff647c291fdda92596c0ef9 P 8fba367aecce477ebf2490ce10206f90 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8fba367aecce477ebf2490ce10206f90 (127.0.206.67). New cstate: current_term: 1 leader_uuid: "8fba367aecce477ebf2490ce10206f90" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ddd9f3878b1a46e0a8cb9b64cd38f40c" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 39203 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "75531f2c748944968122e95dc7df2f56" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 38719 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8fba367aecce477ebf2490ce10206f90" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 42943 } health_report { overall_health: HEALTHY } } }
I20260619 10:17:26.033141 5881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:26.031881 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41804 (local address 127.0.206.67:42943)
0619 10:17:26.031945 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0619 10:17:26.031947 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 10:17:26.031957 (+ 10us) server_negotiation.cc:407] Connection header received
0619 10:17:26.032071 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:17:26.032073 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:17:26.032093 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:17:26.032159 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:17:26.032780 (+ 621us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 10:17:26.032781 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 10:17:26.032782 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 10:17:26.032786 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0619 10:17:26.032801 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 10:17:26.032805 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 10:17:26.032872 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0619 10:17:26.032975 (+ 103us) server_negotiation.cc:299] Negotiation successful
0619 10:17:26.032994 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260619 10:17:26.035126 5692 raft_consensus.cc:1261] T b8e7521ddff647c291fdda92596c0ef9 P ddd9f3878b1a46e0a8cb9b64cd38f40c [term 1 FOLLOWER]: Refusing update from remote peer 8fba367aecce477ebf2490ce10206f90: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 10:17:26.035126 5561 raft_consensus.cc:1261] T b8e7521ddff647c291fdda92596c0ef9 P 75531f2c748944968122e95dc7df2f56 [term 1 FOLLOWER]: Refusing update from remote peer 8fba367aecce477ebf2490ce10206f90: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 10:17:26.035459 5891 consensus_queue.cc:1035] T b8e7521ddff647c291fdda92596c0ef9 P 8fba367aecce477ebf2490ce10206f90 [LEADER]: Connected to new peer: Peer: permanent_uuid: "75531f2c748944968122e95dc7df2f56" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 38719 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 10:17:26.035617 5893 consensus_queue.cc:1035] T b8e7521ddff647c291fdda92596c0ef9 P 8fba367aecce477ebf2490ce10206f90 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ddd9f3878b1a46e0a8cb9b64cd38f40c" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 39203 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 10:17:26.038666 5898 mvcc.cc:205] Tried to move back new op lower bound from 7298515951756836864 to 7298515951617277952. Current Snapshot: MvccSnapshot[applied={T|T < 7298515951756836864}]
I20260619 10:17:26.038937 5897 mvcc.cc:205] Tried to move back new op lower bound from 7298515951756836864 to 7298515951617277952. Current Snapshot: MvccSnapshot[applied={T|T < 7298515951756836864 or (T in {7298515951756836864})}]
I20260619 10:17:26.039227 5896 mvcc.cc:205] Tried to move back new op lower bound from 7298515951756836864 to 7298515951617277952. Current Snapshot: MvccSnapshot[applied={T|T < 7298515951756836864 or (T in {7298515951756836864})}]
I20260619 10:17:26.040231 5418 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:36156:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 10:17:26.040318 5418 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:36156:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 10:17:26.041054 5418 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ff3e5dd931fb408ab774a9ac881109ae: Sending DeleteTablet for 3 replicas of tablet b8e7521ddff647c291fdda92596c0ef9
I20260619 10:17:26.041349 5672 tablet_service.cc:1514] Processing DeleteTablet for tablet b8e7521ddff647c291fdda92596c0ef9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 10:17:26 UTC) from {username='slave'} at 127.0.0.1:41066
I20260619 10:17:26.041389 5541 tablet_service.cc:1514] Processing DeleteTablet for tablet b8e7521ddff647c291fdda92596c0ef9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 10:17:26 UTC) from {username='slave'} at 127.0.0.1:43996
I20260619 10:17:26.041585 5906 tablet_replica.cc:330] T b8e7521ddff647c291fdda92596c0ef9 P 75531f2c748944968122e95dc7df2f56: stopping tablet replica
I20260619 10:17:26.041579 5803 tablet_service.cc:1514] Processing DeleteTablet for tablet b8e7521ddff647c291fdda92596c0ef9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 10:17:26 UTC) from {username='slave'} at 127.0.0.1:41792
I20260619 10:17:26.041658 825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTU9OP6/build/release/bin/kudu with pid 5479
I20260619 10:17:26.041676 5906 raft_consensus.cc:2229] T b8e7521ddff647c291fdda92596c0ef9 P 75531f2c748944968122e95dc7df2f56 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 10:17:26.042382 5905 tablet_replica.cc:330] T b8e7521ddff647c291fdda92596c0ef9 P ddd9f3878b1a46e0a8cb9b64cd38f40c: stopping tablet replica
I20260619 10:17:26.042526 5905 raft_consensus.cc:2229] T b8e7521ddff647c291fdda92596c0ef9 P ddd9f3878b1a46e0a8cb9b64cd38f40c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 10:17:26.042709 5905 raft_consensus.cc:2258] T b8e7521ddff647c291fdda92596c0ef9 P ddd9f3878b1a46e0a8cb9b64cd38f40c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 10:17:26.043212 5905 ts_tablet_manager.cc:1905] T b8e7521ddff647c291fdda92596c0ef9 P ddd9f3878b1a46e0a8cb9b64cd38f40c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 10:17:26.044932 5905 ts_tablet_manager.cc:1918] T b8e7521ddff647c291fdda92596c0ef9 P ddd9f3878b1a46e0a8cb9b64cd38f40c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 10:17:26.045063 5905 log.cc:1196] T b8e7521ddff647c291fdda92596c0ef9 P ddd9f3878b1a46e0a8cb9b64cd38f40c: Deleting WAL directory at /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781864224577058-825-0/minicluster-data/ts-1/wal/wals/b8e7521ddff647c291fdda92596c0ef9
I20260619 10:17:26.045405 5905 ts_tablet_manager.cc:1939] T b8e7521ddff647c291fdda92596c0ef9 P ddd9f3878b1a46e0a8cb9b64cd38f40c: Deleting consensus metadata
I20260619 10:17:26.045975 5404 catalog_manager.cc:4841] TS ddd9f3878b1a46e0a8cb9b64cd38f40c (127.0.206.66:39203): tablet b8e7521ddff647c291fdda92596c0ef9 (table test-table [id=ee669a78623d428db4d2f516bd9efe5d]) successfully deleted
I20260619 10:17:26.046716 5907 tablet_replica.cc:330] T b8e7521ddff647c291fdda92596c0ef9 P 8fba367aecce477ebf2490ce10206f90: stopping tablet replica
I20260619 10:17:26.046818 5907 raft_consensus.cc:2229] T b8e7521ddff647c291fdda92596c0ef9 P 8fba367aecce477ebf2490ce10206f90 [term 1 LEADER]: Raft consensus shutting down.
I20260619 10:17:26.046947 5907 raft_consensus.cc:2258] T b8e7521ddff647c291fdda92596c0ef9 P 8fba367aecce477ebf2490ce10206f90 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 10:17:26.047374 5907 ts_tablet_manager.cc:1905] T b8e7521ddff647c291fdda92596c0ef9 P 8fba367aecce477ebf2490ce10206f90: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260619 10:17:26.047724 5403 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.0.206.65:38719 (error 108)
I20260619 10:17:26.048180 5877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:26.048048 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.65:38719 (local address 127.0.0.1:44008)
0619 10:17:26.048108 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0619 10:17:26.048147 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.65:38719: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260619 10:17:26.048225 825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTU9OP6/build/release/bin/kudu with pid 5610
W20260619 10:17:26.048383 5403 catalog_manager.cc:4584] TS 75531f2c748944968122e95dc7df2f56 (127.0.206.65:38719): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet b8e7521ddff647c291fdda92596c0ef9: Network error: Client connection negotiation failed: client connection to 127.0.206.65:38719: connect: Connection refused (error 111)
I20260619 10:17:26.049234 5907 ts_tablet_manager.cc:1918] T b8e7521ddff647c291fdda92596c0ef9 P 8fba367aecce477ebf2490ce10206f90: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 10:17:26.049304 5907 log.cc:1196] T b8e7521ddff647c291fdda92596c0ef9 P 8fba367aecce477ebf2490ce10206f90: Deleting WAL directory at /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781864224577058-825-0/minicluster-data/ts-2/wal/wals/b8e7521ddff647c291fdda92596c0ef9
I20260619 10:17:26.049583 5907 ts_tablet_manager.cc:1939] T b8e7521ddff647c291fdda92596c0ef9 P 8fba367aecce477ebf2490ce10206f90: Deleting consensus metadata
I20260619 10:17:26.050062 5405 catalog_manager.cc:4841] TS 8fba367aecce477ebf2490ce10206f90 (127.0.206.67:42943): tablet b8e7521ddff647c291fdda92596c0ef9 (table test-table [id=ee669a78623d428db4d2f516bd9efe5d]) successfully deleted
I20260619 10:17:26.054023 825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTU9OP6/build/release/bin/kudu with pid 5741
I20260619 10:17:26.058480 825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTU9OP6/build/release/bin/kudu with pid 5388
2026-06-19T10:17:26Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (731 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260619 10:17:26.078013 825 mini_oidc.cc:152] Starting JWKS server
I20260619 10:17:26.079221 825 webserver.cc:460] Webserver started at https://127.0.0.1:42221/ using document root <none> and password file <none>
I20260619 10:17:26.079344 825 mini_oidc.cc:189] Starting OIDC Discovery server
I20260619 10:17:26.079522 825 webserver.cc:460] Webserver started at http://127.0.0.1:46475/ using document root <none> and password file <none>
2026-06-19T10:17:26Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T10:17:26Z Disabled control of system clock
I20260619 10:17:26.087061 825 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTU9OP6/build/release/bin/kudu
/tmp/dist-test-taskTU9OP6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781864224577058-825-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781864224577058-825-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781864224577058-825-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781864224577058-825-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.206.126:44589
--webserver_interface=127.0.206.126
--webserver_port=0
--builtin_ntp_servers=127.0.206.84:33535
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.206.126:44589
--enable_jwt_token_auth=true
--jwks_url=https://localhost:42221/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781864224577058-825-0/testchainca.cert with env {}
W20260619 10:17:26.166249 5914 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 10:17:26.166448 5914 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 10:17:26.166471 5914 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 10:17:26.167604 5914 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 10:17:26.167641 5914 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 10:17:26.167654 5914 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 10:17:26.167667 5914 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 10:17:26.167680 5914 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260619 10:17:26.167691 5914 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260619 10:17:26.167703 5914 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:42221/jwks.json
I20260619 10:17:26.169149 5914 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.206.84:33535
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781864224577058-825-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781864224577058-825-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.206.126:44589
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.206.126:44589
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:42221/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781864224577058-825-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.206.126
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781864224577058-825-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5914
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781864224577058-825-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 09:32:44 UTC on ccycloud.quasar-gbtldp.root.comops.site
build id 4455440
I20260619 10:17:26.169335 5914 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 10:17:26.169489 5914 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 10:17:26.174263 5922 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 10:17:26.174273 5920 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 10:17:26.174284 5919 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 10:17:26.174510 5914 server_base.cc:1034] running on GCE node
I20260619 10:17:26.174798 5914 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 10:17:26.175124 5914 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 10:17:26.176263 5914 hybrid_clock.cc:648] HybridClock initialized: now 1781864246176252 us; error 29 us; skew 500 ppm
I20260619 10:17:26.177390 5914 webserver.cc:460] Webserver started at http://127.0.206.126:35329/ using document root <none> and password file <none>
I20260619 10:17:26.177637 5914 fs_manager.cc:359] Metadata directory not provided
I20260619 10:17:26.177681 5914 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 10:17:26.177846 5914 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 10:17:26.178952 5914 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781864224577058-825-0/minicluster-data/master-0/data/instance:
uuid: "888ca2dbc2714f3ca19024dd8b4d0f84"
format_stamp: "Formatted at 2026-06-19 10:17:26 on dist-test-slave-6pt2"
server_key: "8d05483d5d02322e6fbe5a6d97fc7e9d"
server_key_iv: "f0f997294712c9297784d1abbf5fdda3"
server_key_version: "encryptionkey@0"
I20260619 10:17:26.179329 5914 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781864224577058-825-0/minicluster-data/master-0/wal/instance:
uuid: "888ca2dbc2714f3ca19024dd8b4d0f84"
format_stamp: "Formatted at 2026-06-19 10:17:26 on dist-test-slave-6pt2"
server_key: "8d05483d5d02322e6fbe5a6d97fc7e9d"
server_key_iv: "f0f997294712c9297784d1abbf5fdda3"
server_key_version: "encryptionkey@0"
I20260619 10:17:26.180579 5914 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 10:17:26.181340 5928 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 10:17:26.181507 5914 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 10:17:26.181598 5914 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781864224577058-825-0/minicluster-data/master-0/data,/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781864224577058-825-0/minicluster-data/master-0/wal
uuid: "888ca2dbc2714f3ca19024dd8b4d0f84"
format_stamp: "Formatted at 2026-06-19 10:17:26 on dist-test-slave-6pt2"
server_key: "8d05483d5d02322e6fbe5a6d97fc7e9d"
server_key_iv: "f0f997294712c9297784d1abbf5fdda3"
server_key_version: "encryptionkey@0"
I20260619 10:17:26.181677 5914 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781864224577058-825-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781864224577058-825-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781864224577058-825-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 10:17:26.193450 5914 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 10:17:26.193729 5914 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 10:17:26.193867 5914 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 10:17:26.197522 5914 rpc_server.cc:307] RPC server started. Bound to: 127.0.206.126:44589
I20260619 10:17:26.197559 5980 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.206.126:44589 every 8 connection(s)
I20260619 10:17:26.197883 5914 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781864224577058-825-0/minicluster-data/master-0/data/info.pb
I20260619 10:17:26.198952 5981 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 10:17:26.201097 825 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTU9OP6/build/release/bin/kudu as pid 5914
I20260619 10:17:26.201143 5981 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 888ca2dbc2714f3ca19024dd8b4d0f84: Bootstrap starting.
I20260619 10:17:26.201197 825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781864224577058-825-0/minicluster-data/master-0/wal/instance
I20260619 10:17:26.201371 825 external_mini_cluster.cc:1428] Setting key a72f62177728180445947047bdd654b7
I20260619 10:17:26.201874 5981 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 888ca2dbc2714f3ca19024dd8b4d0f84: Neither blocks nor log segments found. Creating new log.
I20260619 10:17:26.202209 5981 log.cc:824] T 00000000000000000000000000000000 P 888ca2dbc2714f3ca19024dd8b4d0f84: Log is configured to *not* fsync() on all Append() calls
I20260619 10:17:26.202958 5981 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 888ca2dbc2714f3ca19024dd8b4d0f84: No bootstrap required, opened a new log
I20260619 10:17:26.204928 5981 raft_consensus.cc:348] T 00000000000000000000000000000000 P 888ca2dbc2714f3ca19024dd8b4d0f84 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "888ca2dbc2714f3ca19024dd8b4d0f84" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 44589 } }
I20260619 10:17:26.205104 5981 raft_consensus.cc:374] T 00000000000000000000000000000000 P 888ca2dbc2714f3ca19024dd8b4d0f84 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 10:17:26.205148 5981 raft_consensus.cc:729] T 00000000000000000000000000000000 P 888ca2dbc2714f3ca19024dd8b4d0f84 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 888ca2dbc2714f3ca19024dd8b4d0f84, State: Initialized, Role: FOLLOWER
I20260619 10:17:26.205286 5981 consensus_queue.cc:260] T 00000000000000000000000000000000 P 888ca2dbc2714f3ca19024dd8b4d0f84 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "888ca2dbc2714f3ca19024dd8b4d0f84" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 44589 } }
I20260619 10:17:26.205346 5981 raft_consensus.cc:388] T 00000000000000000000000000000000 P 888ca2dbc2714f3ca19024dd8b4d0f84 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 10:17:26.205367 5981 raft_consensus.cc:482] T 00000000000000000000000000000000 P 888ca2dbc2714f3ca19024dd8b4d0f84 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 10:17:26.205433 5981 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 888ca2dbc2714f3ca19024dd8b4d0f84 [term 0 FOLLOWER]: Advancing to term 1
I20260619 10:17:26.206060 5981 raft_consensus.cc:504] T 00000000000000000000000000000000 P 888ca2dbc2714f3ca19024dd8b4d0f84 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "888ca2dbc2714f3ca19024dd8b4d0f84" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 44589 } }
I20260619 10:17:26.206156 5981 leader_election.cc:302] T 00000000000000000000000000000000 P 888ca2dbc2714f3ca19024dd8b4d0f84 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 888ca2dbc2714f3ca19024dd8b4d0f84; no voters:
I20260619 10:17:26.206318 5981 leader_election.cc:288] T 00000000000000000000000000000000 P 888ca2dbc2714f3ca19024dd8b4d0f84 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 10:17:26.206449 5986 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 888ca2dbc2714f3ca19024dd8b4d0f84 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 10:17:26.206581 5986 raft_consensus.cc:686] T 00000000000000000000000000000000 P 888ca2dbc2714f3ca19024dd8b4d0f84 [term 1 LEADER]: Becoming Leader. State: Replica: 888ca2dbc2714f3ca19024dd8b4d0f84, State: Running, Role: LEADER
I20260619 10:17:26.206589 5981 sys_catalog.cc:564] T 00000000000000000000000000000000 P 888ca2dbc2714f3ca19024dd8b4d0f84 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 10:17:26.206696 5986 consensus_queue.cc:237] T 00000000000000000000000000000000 P 888ca2dbc2714f3ca19024dd8b4d0f84 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "888ca2dbc2714f3ca19024dd8b4d0f84" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 44589 } }
I20260619 10:17:26.207006 5987 sys_catalog.cc:455] T 00000000000000000000000000000000 P 888ca2dbc2714f3ca19024dd8b4d0f84 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "888ca2dbc2714f3ca19024dd8b4d0f84" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "888ca2dbc2714f3ca19024dd8b4d0f84" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 44589 } } }
I20260619 10:17:26.207090 5987 sys_catalog.cc:458] T 00000000000000000000000000000000 P 888ca2dbc2714f3ca19024dd8b4d0f84 [sys.catalog]: This master's current role is: LEADER
I20260619 10:17:26.206991 5986 sys_catalog.cc:455] T 00000000000000000000000000000000 P 888ca2dbc2714f3ca19024dd8b4d0f84 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 888ca2dbc2714f3ca19024dd8b4d0f84. Latest consensus state: current_term: 1 leader_uuid: "888ca2dbc2714f3ca19024dd8b4d0f84" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "888ca2dbc2714f3ca19024dd8b4d0f84" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 44589 } } }
I20260619 10:17:26.207290 5986 sys_catalog.cc:458] T 00000000000000000000000000000000 P 888ca2dbc2714f3ca19024dd8b4d0f84 [sys.catalog]: This master's current role is: LEADER
I20260619 10:17:26.207414 5992 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 10:17:26.207451 5984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:26.201851 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51852 (local address 127.0.206.126:44589)
0619 10:17:26.202079 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0619 10:17:26.202088 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 10:17:26.202122 (+ 34us) server_negotiation.cc:407] Connection header received
0619 10:17:26.202659 (+ 537us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:17:26.202668 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:17:26.202741 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:17:26.202853 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:17:26.203103 (+ 250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:26.203977 (+ 874us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:26.205137 (+ 1160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:26.205709 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:26.205798 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:26.206620 (+ 822us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 10:17:26.206624 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 10:17:26.206625 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 10:17:26.206854 (+ 229us) server_negotiation.cc:922] calling sasl_server_start()
0619 10:17:26.206902 (+ 48us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 10:17:26.206916 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 10:17:26.206956 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0619 10:17:26.207093 (+ 137us) server_negotiation.cc:299] Negotiation successful
0619 10:17:26.207178 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":134,"thread_start_us":70,"threads_started":1}
I20260619 10:17:26.207872 5992 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 10:17:26.208470 825 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 10:17:26.209365 5992 catalog_manager.cc:1347] Generated new cluster ID: 7b9a9893e27446bc9cff14ba39e0f491
I20260619 10:17:26.209415 5992 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 10:17:26.214450 5992 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 10:17:26.215142 5992 catalog_manager.cc:1504] Loading token signing keys...
I20260619 10:17:26.220645 5992 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 888ca2dbc2714f3ca19024dd8b4d0f84: Generated new TSK 0
I20260619 10:17:26.220813 5992 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260619 10:17:26.243620 6011 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260619 10:17:26.243726 6011 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260619 10:17:26.244033 5984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:26.228940 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51868 (local address 127.0.206.126:44589)
0619 10:17:26.228983 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0619 10:17:26.228985 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 10:17:26.229120 (+ 135us) server_negotiation.cc:407] Connection header received
0619 10:17:26.229210 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:17:26.229211 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:17:26.229220 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:17:26.229274 (+ 54us) server_negotiation.cc:226] Negotiated authn=JWT
0619 10:17:26.229415 (+ 141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:26.229887 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:26.231101 (+ 1214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:26.231299 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:26.231349 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:26.231572 (+ 223us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0619 10:17:26.243802 (+ 12230us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42221/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0619 10:17:26.243958 (+ 156us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:51868: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42221/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":23}
W20260619 10:17:26.244094 5984 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:51868: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42221/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
W20260619 10:17:26.244333 6010 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 10:17:26.228840 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:44589 (local address 127.0.0.1:51868)
0619 10:17:26.229038 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0619 10:17:26.229050 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 10:17:26.229123 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 10:17:26.229288 (+ 165us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 10:17:26.229289 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 10:17:26.229291 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0619 10:17:26.229359 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:26.229361 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:26.229976 (+ 615us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:26.229976 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:26.230996 (+ 1020us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:26.231001 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:26.231375 (+ 374us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:26.231377 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:26.231439 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:26.231461 (+ 22us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0619 10:17:26.243910 (+ 12449us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42221/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0619 10:17:26.244007 (+ 97us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.0.206.126:44589: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42221/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":108,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.0.206.126:44589: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42221/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260619 10:17:26.244748 825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTU9OP6/build/release/bin/kudu with pid 5914
2026-06-19T10:17:26Z chronyd exiting
I20260619 10:17:26.280553 825 test_util.cc:175] -----------------------------------------------
I20260619 10:17:26.280647 825 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781864224577058-825-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (203 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260619 10:17:26.281436 825 mini_oidc.cc:152] Starting JWKS server
I20260619 10:17:26.282110 825 webserver.cc:460] Webserver started at https://127.0.0.1:36461/ using document root <none> and password file <none>
I20260619 10:17:26.282182 825 mini_oidc.cc:189] Starting OIDC Discovery server
I20260619 10:17:26.282359 825 webserver.cc:460] Webserver started at http://127.0.0.1:40785/ using document root <none> and password file <none>
2026-06-19T10:17:26Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T10:17:26Z Disabled control of system clock
I20260619 10:17:26.289503 825 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTU9OP6/build/release/bin/kudu
/tmp/dist-test-taskTU9OP6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781864224577058-825-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781864224577058-825-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781864224577058-825-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781864224577058-825-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.206.126:46511
--webserver_interface=127.0.206.126
--webserver_port=0
--builtin_ntp_servers=127.0.206.84:38747
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.206.126:46511
--enable_jwt_token_auth=true
--jwks_url=https://localhost:36461/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781864224577058-825-0/testchainca.cert with env {}
W20260619 10:17:26.367053 6017 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 10:17:26.367247 6017 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 10:17:26.367268 6017 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 10:17:26.368497 6017 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 10:17:26.368541 6017 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 10:17:26.368553 6017 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 10:17:26.368566 6017 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 10:17:26.368577 6017 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260619 10:17:26.368588 6017 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260619 10:17:26.368599 6017 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:36461/jwks.json
I20260619 10:17:26.370038 6017 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.206.84:38747
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781864224577058-825-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781864224577058-825-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.206.126:46511
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.206.126:46511
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:36461/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781864224577058-825-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.206.126
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781864224577058-825-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6017
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781864224577058-825-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 09:32:44 UTC on ccycloud.quasar-gbtldp.root.comops.site
build id 4455440
I20260619 10:17:26.370249 6017 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 10:17:26.370441 6017 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 10:17:26.375149 6022 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 10:17:26.375144 6023 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 10:17:26.375224 6025 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 10:17:26.375334 6017 server_base.cc:1034] running on GCE node
I20260619 10:17:26.375614 6017 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 10:17:26.375900 6017 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 10:17:26.377032 6017 hybrid_clock.cc:648] HybridClock initialized: now 1781864246377021 us; error 26 us; skew 500 ppm
I20260619 10:17:26.378124 6017 webserver.cc:460] Webserver started at http://127.0.206.126:33101/ using document root <none> and password file <none>
I20260619 10:17:26.378364 6017 fs_manager.cc:359] Metadata directory not provided
I20260619 10:17:26.378407 6017 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 10:17:26.378547 6017 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 10:17:26.379601 6017 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781864224577058-825-0/minicluster-data/master-0/data/instance:
uuid: "0852faf13e674733a4600aa2a1dc9724"
format_stamp: "Formatted at 2026-06-19 10:17:26 on dist-test-slave-6pt2"
server_key: "c2c73618d62561f21c5d947d501e44f6"
server_key_iv: "45caf26ace1d383c8de8fe1acac9a6f4"
server_key_version: "encryptionkey@0"
I20260619 10:17:26.379948 6017 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781864224577058-825-0/minicluster-data/master-0/wal/instance:
uuid: "0852faf13e674733a4600aa2a1dc9724"
format_stamp: "Formatted at 2026-06-19 10:17:26 on dist-test-slave-6pt2"
server_key: "c2c73618d62561f21c5d947d501e44f6"
server_key_iv: "45caf26ace1d383c8de8fe1acac9a6f4"
server_key_version: "encryptionkey@0"
I20260619 10:17:26.381160 6017 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 10:17:26.381937 6031 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 10:17:26.382159 6017 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 10:17:26.382258 6017 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781864224577058-825-0/minicluster-data/master-0/data,/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781864224577058-825-0/minicluster-data/master-0/wal
uuid: "0852faf13e674733a4600aa2a1dc9724"
format_stamp: "Formatted at 2026-06-19 10:17:26 on dist-test-slave-6pt2"
server_key: "c2c73618d62561f21c5d947d501e44f6"
server_key_iv: "45caf26ace1d383c8de8fe1acac9a6f4"
server_key_version: "encryptionkey@0"
I20260619 10:17:26.382331 6017 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781864224577058-825-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781864224577058-825-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781864224577058-825-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 10:17:26.413048 6017 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 10:17:26.413388 6017 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 10:17:26.413544 6017 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 10:17:26.417467 6017 rpc_server.cc:307] RPC server started. Bound to: 127.0.206.126:46511
I20260619 10:17:26.417531 6083 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.206.126:46511 every 8 connection(s)
I20260619 10:17:26.417858 6017 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781864224577058-825-0/minicluster-data/master-0/data/info.pb
I20260619 10:17:26.418876 6084 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 10:17:26.421073 6084 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0852faf13e674733a4600aa2a1dc9724: Bootstrap starting.
I20260619 10:17:26.421644 6084 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0852faf13e674733a4600aa2a1dc9724: Neither blocks nor log segments found. Creating new log.
I20260619 10:17:26.421947 6084 log.cc:824] T 00000000000000000000000000000000 P 0852faf13e674733a4600aa2a1dc9724: Log is configured to *not* fsync() on all Append() calls
I20260619 10:17:26.422561 6084 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0852faf13e674733a4600aa2a1dc9724: No bootstrap required, opened a new log
I20260619 10:17:26.423722 825 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTU9OP6/build/release/bin/kudu as pid 6017
I20260619 10:17:26.423830 825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781864224577058-825-0/minicluster-data/master-0/wal/instance
I20260619 10:17:26.424015 825 external_mini_cluster.cc:1428] Setting key e8ed1c32fc0f4bd83677be577a346edc
I20260619 10:17:26.424568 6084 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0852faf13e674733a4600aa2a1dc9724 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0852faf13e674733a4600aa2a1dc9724" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 46511 } }
I20260619 10:17:26.424733 6084 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0852faf13e674733a4600aa2a1dc9724 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 10:17:26.424759 6084 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0852faf13e674733a4600aa2a1dc9724 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0852faf13e674733a4600aa2a1dc9724, State: Initialized, Role: FOLLOWER
I20260619 10:17:26.424899 6084 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0852faf13e674733a4600aa2a1dc9724 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0852faf13e674733a4600aa2a1dc9724" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 46511 } }
I20260619 10:17:26.424969 6084 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0852faf13e674733a4600aa2a1dc9724 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 10:17:26.425009 6084 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0852faf13e674733a4600aa2a1dc9724 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 10:17:26.425057 6084 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0852faf13e674733a4600aa2a1dc9724 [term 0 FOLLOWER]: Advancing to term 1
I20260619 10:17:26.425634 6084 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0852faf13e674733a4600aa2a1dc9724 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0852faf13e674733a4600aa2a1dc9724" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 46511 } }
I20260619 10:17:26.425719 6084 leader_election.cc:302] T 00000000000000000000000000000000 P 0852faf13e674733a4600aa2a1dc9724 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0852faf13e674733a4600aa2a1dc9724; no voters:
I20260619 10:17:26.425900 6084 leader_election.cc:288] T 00000000000000000000000000000000 P 0852faf13e674733a4600aa2a1dc9724 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 10:17:26.425927 6089 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0852faf13e674733a4600aa2a1dc9724 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 10:17:26.426045 6089 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0852faf13e674733a4600aa2a1dc9724 [term 1 LEADER]: Becoming Leader. State: Replica: 0852faf13e674733a4600aa2a1dc9724, State: Running, Role: LEADER
I20260619 10:17:26.426126 6089 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0852faf13e674733a4600aa2a1dc9724 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0852faf13e674733a4600aa2a1dc9724" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 46511 } }
I20260619 10:17:26.426182 6084 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0852faf13e674733a4600aa2a1dc9724 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 10:17:26.426416 6089 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0852faf13e674733a4600aa2a1dc9724 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0852faf13e674733a4600aa2a1dc9724. Latest consensus state: current_term: 1 leader_uuid: "0852faf13e674733a4600aa2a1dc9724" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0852faf13e674733a4600aa2a1dc9724" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 46511 } } }
I20260619 10:17:26.426491 6089 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0852faf13e674733a4600aa2a1dc9724 [sys.catalog]: This master's current role is: LEADER
I20260619 10:17:26.426429 6090 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0852faf13e674733a4600aa2a1dc9724 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0852faf13e674733a4600aa2a1dc9724" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0852faf13e674733a4600aa2a1dc9724" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 46511 } } }
I20260619 10:17:26.426525 6090 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0852faf13e674733a4600aa2a1dc9724 [sys.catalog]: This master's current role is: LEADER
I20260619 10:17:26.426829 6093 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 10:17:26.427350 6093 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 10:17:26.429549 6093 catalog_manager.cc:1347] Generated new cluster ID: a773243760d7432e891a57ce5687c9ef
I20260619 10:17:26.429633 6093 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 10:17:26.430078 6088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:26.424525 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38510 (local address 127.0.206.126:46511)
0619 10:17:26.424716 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0619 10:17:26.424723 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 10:17:26.424768 (+ 45us) server_negotiation.cc:407] Connection header received
0619 10:17:26.424891 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:17:26.424896 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:17:26.424946 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:17:26.425089 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:17:26.425293 (+ 204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:26.426915 (+ 1622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:26.428031 (+ 1116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:26.428404 (+ 373us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:26.428483 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:26.429253 (+ 770us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 10:17:26.429257 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 10:17:26.429257 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 10:17:26.429453 (+ 196us) server_negotiation.cc:922] calling sasl_server_start()
0619 10:17:26.429493 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 10:17:26.429508 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 10:17:26.429585 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0619 10:17:26.429796 (+ 211us) server_negotiation.cc:299] Negotiation successful
0619 10:17:26.429874 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":122,"thread_start_us":90,"threads_started":1}
I20260619 10:17:26.430984 825 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 10:17:26.434785 6093 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 10:17:26.435376 6093 catalog_manager.cc:1504] Loading token signing keys...
I20260619 10:17:26.440039 6093 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0852faf13e674733a4600aa2a1dc9724: Generated new TSK 0
I20260619 10:17:26.440196 6093 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260619 10:17:26.464835 6114 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260619 10:17:26.464919 6114 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260619 10:17:26.465219 6088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:26.451133 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38514 (local address 127.0.206.126:46511)
0619 10:17:26.451180 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0619 10:17:26.451182 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 10:17:26.451279 (+ 97us) server_negotiation.cc:407] Connection header received
0619 10:17:26.451346 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:17:26.451346 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:17:26.451355 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:17:26.451412 (+ 57us) server_negotiation.cc:226] Negotiated authn=JWT
0619 10:17:26.451558 (+ 146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:26.452024 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:26.453144 (+ 1120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:26.453315 (+ 171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:26.453362 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:26.453535 (+ 173us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0619 10:17:26.465048 (+ 11513us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36461/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0619 10:17:26.465171 (+ 123us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:38514: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36461/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":26}
W20260619 10:17:26.465287 6088 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:38514: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36461/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
W20260619 10:17:26.465384 6113 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 10:17:26.451049 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:46511 (local address 127.0.0.1:38514)
0619 10:17:26.451199 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0619 10:17:26.451211 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 10:17:26.451281 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 10:17:26.451428 (+ 147us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 10:17:26.451430 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 10:17:26.451431 (+ 1us) client_negotiation.cc:189] Negotiated authn=JWT
0619 10:17:26.451493 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:26.451496 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:26.452109 (+ 613us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:26.452110 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:26.453066 (+ 956us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:26.453069 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:26.453381 (+ 312us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:26.453382 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:26.453440 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:26.453446 (+ 6us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0619 10:17:26.465161 (+ 11715us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36461/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0619 10:17:26.465281 (+ 120us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.0.206.126:46511: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36461/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":87,"threads_started":1}
I20260619 10:17:26.465708 825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTU9OP6/build/release/bin/kudu with pid 6017
2026-06-19T10:17:26Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (221 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260619 10:17:26.503103 825 mini_oidc.cc:152] Starting JWKS server
I20260619 10:17:26.503782 825 webserver.cc:460] Webserver started at https://127.0.0.1:44935/ using document root <none> and password file <none>
I20260619 10:17:26.503849 825 mini_oidc.cc:189] Starting OIDC Discovery server
I20260619 10:17:26.504026 825 webserver.cc:460] Webserver started at http://127.0.0.1:45845/ using document root <none> and password file <none>
2026-06-19T10:17:26Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T10:17:26Z Disabled control of system clock
I20260619 10:17:26.511440 825 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTU9OP6/build/release/bin/kudu
/tmp/dist-test-taskTU9OP6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781864224577058-825-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781864224577058-825-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781864224577058-825-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781864224577058-825-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.206.126:42533
--webserver_interface=127.0.206.126
--webserver_port=0
--builtin_ntp_servers=127.0.206.84:39573
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.206.126:42533
--enable_jwt_token_auth=true
--jwks_url=https://localhost:44935/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260619 10:17:26.589342 6120 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 10:17:26.589542 6120 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 10:17:26.589563 6120 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 10:17:26.590706 6120 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 10:17:26.590740 6120 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 10:17:26.590752 6120 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 10:17:26.590765 6120 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 10:17:26.590777 6120 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260619 10:17:26.590790 6120 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260619 10:17:26.590812 6120 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:44935/jwks.json
I20260619 10:17:26.592092 6120 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.206.84:39573
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781864224577058-825-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781864224577058-825-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.206.126:42533
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.206.126:42533
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:44935/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781864224577058-825-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.206.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6120
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781864224577058-825-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 09:32:44 UTC on ccycloud.quasar-gbtldp.root.comops.site
build id 4455440
I20260619 10:17:26.592285 6120 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 10:17:26.592504 6120 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 10:17:26.597143 6125 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 10:17:26.597159 6126 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 10:17:26.597178 6128 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 10:17:26.597419 6120 server_base.cc:1034] running on GCE node
I20260619 10:17:26.597671 6120 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 10:17:26.597966 6120 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 10:17:26.599123 6120 hybrid_clock.cc:648] HybridClock initialized: now 1781864246599109 us; error 25 us; skew 500 ppm
I20260619 10:17:26.600271 6120 webserver.cc:460] Webserver started at http://127.0.206.126:39797/ using document root <none> and password file <none>
I20260619 10:17:26.600507 6120 fs_manager.cc:359] Metadata directory not provided
I20260619 10:17:26.600574 6120 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 10:17:26.600706 6120 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 10:17:26.601728 6120 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781864224577058-825-0/minicluster-data/master-0/data/instance:
uuid: "3e0acfde51f04be2ac67553eb7ead45b"
format_stamp: "Formatted at 2026-06-19 10:17:26 on dist-test-slave-6pt2"
server_key: "1d6f2a036706e02c2fe13235eb10edde"
server_key_iv: "0a56cfc14da405c0b95e9daebbcd67f0"
server_key_version: "encryptionkey@0"
I20260619 10:17:26.602108 6120 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781864224577058-825-0/minicluster-data/master-0/wal/instance:
uuid: "3e0acfde51f04be2ac67553eb7ead45b"
format_stamp: "Formatted at 2026-06-19 10:17:26 on dist-test-slave-6pt2"
server_key: "1d6f2a036706e02c2fe13235eb10edde"
server_key_iv: "0a56cfc14da405c0b95e9daebbcd67f0"
server_key_version: "encryptionkey@0"
I20260619 10:17:26.603428 6120 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 10:17:26.604118 6134 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 10:17:26.604287 6120 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 10:17:26.604388 6120 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781864224577058-825-0/minicluster-data/master-0/data,/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781864224577058-825-0/minicluster-data/master-0/wal
uuid: "3e0acfde51f04be2ac67553eb7ead45b"
format_stamp: "Formatted at 2026-06-19 10:17:26 on dist-test-slave-6pt2"
server_key: "1d6f2a036706e02c2fe13235eb10edde"
server_key_iv: "0a56cfc14da405c0b95e9daebbcd67f0"
server_key_version: "encryptionkey@0"
I20260619 10:17:26.604460 6120 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781864224577058-825-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781864224577058-825-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781864224577058-825-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 10:17:26.626819 6120 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 10:17:26.627125 6120 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 10:17:26.627274 6120 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 10:17:26.630995 6120 rpc_server.cc:307] RPC server started. Bound to: 127.0.206.126:42533
I20260619 10:17:26.631026 6186 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.206.126:42533 every 8 connection(s)
I20260619 10:17:26.631379 6120 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781864224577058-825-0/minicluster-data/master-0/data/info.pb
I20260619 10:17:26.632117 6187 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 10:17:26.634737 6187 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3e0acfde51f04be2ac67553eb7ead45b: Bootstrap starting.
I20260619 10:17:26.635308 6187 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3e0acfde51f04be2ac67553eb7ead45b: Neither blocks nor log segments found. Creating new log.
I20260619 10:17:26.635622 6187 log.cc:824] T 00000000000000000000000000000000 P 3e0acfde51f04be2ac67553eb7ead45b: Log is configured to *not* fsync() on all Append() calls
I20260619 10:17:26.635744 825 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTU9OP6/build/release/bin/kudu as pid 6120
I20260619 10:17:26.635867 825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781864224577058-825-0/minicluster-data/master-0/wal/instance
I20260619 10:17:26.636078 825 external_mini_cluster.cc:1428] Setting key 374500294d2cca0605cb181fc13ac7f4
I20260619 10:17:26.636379 6187 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3e0acfde51f04be2ac67553eb7ead45b: No bootstrap required, opened a new log
I20260619 10:17:26.638108 6187 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3e0acfde51f04be2ac67553eb7ead45b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3e0acfde51f04be2ac67553eb7ead45b" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 42533 } }
I20260619 10:17:26.638270 6187 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3e0acfde51f04be2ac67553eb7ead45b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 10:17:26.638352 6187 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3e0acfde51f04be2ac67553eb7ead45b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e0acfde51f04be2ac67553eb7ead45b, State: Initialized, Role: FOLLOWER
I20260619 10:17:26.638514 6187 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3e0acfde51f04be2ac67553eb7ead45b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3e0acfde51f04be2ac67553eb7ead45b" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 42533 } }
I20260619 10:17:26.638634 6187 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3e0acfde51f04be2ac67553eb7ead45b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 10:17:26.638686 6187 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3e0acfde51f04be2ac67553eb7ead45b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 10:17:26.638737 6187 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3e0acfde51f04be2ac67553eb7ead45b [term 0 FOLLOWER]: Advancing to term 1
I20260619 10:17:26.639451 6187 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3e0acfde51f04be2ac67553eb7ead45b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3e0acfde51f04be2ac67553eb7ead45b" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 42533 } }
I20260619 10:17:26.639585 6187 leader_election.cc:302] T 00000000000000000000000000000000 P 3e0acfde51f04be2ac67553eb7ead45b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3e0acfde51f04be2ac67553eb7ead45b; no voters:
I20260619 10:17:26.639789 6187 leader_election.cc:288] T 00000000000000000000000000000000 P 3e0acfde51f04be2ac67553eb7ead45b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 10:17:26.639818 6192 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3e0acfde51f04be2ac67553eb7ead45b [term 1 FOLLOWER]: Leader election won for term 1
I20260619 10:17:26.639923 6192 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3e0acfde51f04be2ac67553eb7ead45b [term 1 LEADER]: Becoming Leader. State: Replica: 3e0acfde51f04be2ac67553eb7ead45b, State: Running, Role: LEADER
I20260619 10:17:26.640053 6187 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3e0acfde51f04be2ac67553eb7ead45b [sys.catalog]: configured and running, proceeding with master startup.
I20260619 10:17:26.640053 6192 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3e0acfde51f04be2ac67553eb7ead45b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3e0acfde51f04be2ac67553eb7ead45b" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 42533 } }
I20260619 10:17:26.640477 6194 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3e0acfde51f04be2ac67553eb7ead45b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3e0acfde51f04be2ac67553eb7ead45b. Latest consensus state: current_term: 1 leader_uuid: "3e0acfde51f04be2ac67553eb7ead45b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3e0acfde51f04be2ac67553eb7ead45b" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 42533 } } }
I20260619 10:17:26.640460 6193 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3e0acfde51f04be2ac67553eb7ead45b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3e0acfde51f04be2ac67553eb7ead45b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3e0acfde51f04be2ac67553eb7ead45b" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 42533 } } }
I20260619 10:17:26.640635 6194 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3e0acfde51f04be2ac67553eb7ead45b [sys.catalog]: This master's current role is: LEADER
I20260619 10:17:26.640722 6193 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3e0acfde51f04be2ac67553eb7ead45b [sys.catalog]: This master's current role is: LEADER
I20260619 10:17:26.640954 6201 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 10:17:26.641556 6201 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 10:17:26.643181 6191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:26.636531 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38754 (local address 127.0.206.126:42533)
0619 10:17:26.636750 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0619 10:17:26.636759 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 10:17:26.636789 (+ 30us) server_negotiation.cc:407] Connection header received
0619 10:17:26.638093 (+ 1304us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:17:26.638100 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:17:26.638156 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:17:26.638288 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:17:26.638482 (+ 194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:26.639490 (+ 1008us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:26.640629 (+ 1139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:26.641258 (+ 629us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:26.641355 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:26.642445 (+ 1090us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 10:17:26.642449 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 10:17:26.642449 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 10:17:26.642674 (+ 225us) server_negotiation.cc:922] calling sasl_server_start()
0619 10:17:26.642723 (+ 49us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 10:17:26.642735 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 10:17:26.642816 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0619 10:17:26.642892 (+ 76us) server_negotiation.cc:299] Negotiation successful
0619 10:17:26.642970 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":140,"thread_start_us":77,"threads_started":1}
I20260619 10:17:26.643184 6201 catalog_manager.cc:1347] Generated new cluster ID: e6d7299fcb6d4e2f9e18cb7614120407
I20260619 10:17:26.643268 6201 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 10:17:26.647454 825 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 10:17:26.687307 6201 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 10:17:26.687857 6201 catalog_manager.cc:1504] Loading token signing keys...
I20260619 10:17:26.696192 6201 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3e0acfde51f04be2ac67553eb7ead45b: Generated new TSK 0
I20260619 10:17:26.696401 6201 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 10:17:26.709213 6191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:26.705585 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38756 (local address 127.0.206.126:42533)
0619 10:17:26.705646 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0619 10:17:26.705648 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 10:17:26.705749 (+ 101us) server_negotiation.cc:407] Connection header received
0619 10:17:26.705791 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:17:26.705792 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:17:26.705805 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:17:26.705887 (+ 82us) server_negotiation.cc:226] Negotiated authn=JWT
0619 10:17:26.706005 (+ 118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:26.706773 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:26.707505 (+ 732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:26.707802 (+ 297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:26.707878 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:26.708017 (+ 139us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0619 10:17:26.709002 (+ 985us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44935/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0619 10:17:26.709129 (+ 127us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:38756: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44935/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":36}
W20260619 10:17:26.709276 6191 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:38756: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44935/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
W20260619 10:17:26.709349 6216 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 10:17:26.705496 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:42533 (local address 127.0.0.1:38756)
0619 10:17:26.705674 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0619 10:17:26.705692 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0619 10:17:26.705743 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 10:17:26.705890 (+ 147us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 10:17:26.705890 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 10:17:26.705892 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0619 10:17:26.705943 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:26.705944 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:26.706845 (+ 901us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:26.706845 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:26.707419 (+ 574us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:26.707421 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:26.707883 (+ 462us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:26.707884 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:26.707921 (+ 37us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:26.707925 (+ 4us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0619 10:17:26.709108 (+ 1183us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44935/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0619 10:17:26.709253 (+ 145us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.0.206.126:42533: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44935/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":103,"threads_started":1}
I20260619 10:17:26.709537 825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTU9OP6/build/release/bin/kudu with pid 6120
E20260619 10:17:26.721468 6217 webserver.cc:533] Webserver: sslize failed:
E20260619 10:17:26.721551 6217 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
2026-06-19T10:17:26Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (244 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-06-19T10:17:26Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T10:17:26Z Disabled control of system clock
I20260619 10:17:26.755311 825 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTU9OP6/build/release/bin/kudu
/tmp/dist-test-taskTU9OP6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781864224577058-825-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781864224577058-825-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781864224577058-825-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781864224577058-825-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.206.126:36263
--webserver_interface=127.0.206.126
--webserver_port=0
--builtin_ntp_servers=127.0.206.84:45869
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.206.126:36263
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781864224577058-825-0/testchainca.cert with env {}
W20260619 10:17:26.832978 6221 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 10:17:26.833209 6221 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 10:17:26.833241 6221 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 10:17:26.834381 6221 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 10:17:26.834451 6221 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 10:17:26.834472 6221 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 10:17:26.834493 6221 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 10:17:26.834520 6221 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260619 10:17:26.834544 6221 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260619 10:17:26.834573 6221 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260619 10:17:26.836018 6221 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.206.84:45869
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781864224577058-825-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781864224577058-825-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.206.126:36263
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.206.126:36263
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781864224577058-825-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.206.126
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781864224577058-825-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6221
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781864224577058-825-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 09:32:44 UTC on ccycloud.quasar-gbtldp.root.comops.site
build id 4455440
I20260619 10:17:26.836282 6221 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 10:17:26.836488 6221 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 10:17:26.841074 6229 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 10:17:26.841100 6227 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 10:17:26.841166 6221 server_base.cc:1034] running on GCE node
W20260619 10:17:26.841101 6226 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 10:17:26.841542 6221 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 10:17:26.841814 6221 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 10:17:26.842947 6221 hybrid_clock.cc:648] HybridClock initialized: now 1781864246842929 us; error 36 us; skew 500 ppm
I20260619 10:17:26.844069 6221 webserver.cc:460] Webserver started at http://127.0.206.126:34165/ using document root <none> and password file <none>
I20260619 10:17:26.844247 6221 fs_manager.cc:359] Metadata directory not provided
I20260619 10:17:26.844285 6221 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 10:17:26.844368 6221 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 10:17:26.845296 6221 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781864224577058-825-0/minicluster-data/master-0/data/instance:
uuid: "46f1ec1bd83846869fa4c6564aadcbac"
format_stamp: "Formatted at 2026-06-19 10:17:26 on dist-test-slave-6pt2"
server_key: "90ae35aca209238154771ef4325dfeef"
server_key_iv: "009b5d434dcc77daa24c021b956b6f61"
server_key_version: "encryptionkey@0"
I20260619 10:17:26.845600 6221 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781864224577058-825-0/minicluster-data/master-0/wal/instance:
uuid: "46f1ec1bd83846869fa4c6564aadcbac"
format_stamp: "Formatted at 2026-06-19 10:17:26 on dist-test-slave-6pt2"
server_key: "90ae35aca209238154771ef4325dfeef"
server_key_iv: "009b5d434dcc77daa24c021b956b6f61"
server_key_version: "encryptionkey@0"
I20260619 10:17:26.846765 6221 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 10:17:26.847435 6235 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 10:17:26.847625 6221 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 10:17:26.847705 6221 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781864224577058-825-0/minicluster-data/master-0/data,/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781864224577058-825-0/minicluster-data/master-0/wal
uuid: "46f1ec1bd83846869fa4c6564aadcbac"
format_stamp: "Formatted at 2026-06-19 10:17:26 on dist-test-slave-6pt2"
server_key: "90ae35aca209238154771ef4325dfeef"
server_key_iv: "009b5d434dcc77daa24c021b956b6f61"
server_key_version: "encryptionkey@0"
I20260619 10:17:26.847757 6221 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781864224577058-825-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781864224577058-825-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781864224577058-825-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 10:17:26.859587 6221 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 10:17:26.859789 6221 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 10:17:26.859898 6221 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 10:17:26.863354 6221 rpc_server.cc:307] RPC server started. Bound to: 127.0.206.126:36263
I20260619 10:17:26.863399 6287 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.206.126:36263 every 8 connection(s)
I20260619 10:17:26.863687 6221 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781864224577058-825-0/minicluster-data/master-0/data/info.pb
I20260619 10:17:26.864532 6288 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 10:17:26.866554 6288 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 46f1ec1bd83846869fa4c6564aadcbac: Bootstrap starting.
I20260619 10:17:26.867112 6288 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 46f1ec1bd83846869fa4c6564aadcbac: Neither blocks nor log segments found. Creating new log.
I20260619 10:17:26.867385 6288 log.cc:824] T 00000000000000000000000000000000 P 46f1ec1bd83846869fa4c6564aadcbac: Log is configured to *not* fsync() on all Append() calls
I20260619 10:17:26.868001 6288 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 46f1ec1bd83846869fa4c6564aadcbac: No bootstrap required, opened a new log
I20260619 10:17:26.869486 6288 raft_consensus.cc:348] T 00000000000000000000000000000000 P 46f1ec1bd83846869fa4c6564aadcbac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "46f1ec1bd83846869fa4c6564aadcbac" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 36263 } }
I20260619 10:17:26.869640 6288 raft_consensus.cc:374] T 00000000000000000000000000000000 P 46f1ec1bd83846869fa4c6564aadcbac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 10:17:26.869688 6288 raft_consensus.cc:729] T 00000000000000000000000000000000 P 46f1ec1bd83846869fa4c6564aadcbac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 46f1ec1bd83846869fa4c6564aadcbac, State: Initialized, Role: FOLLOWER
I20260619 10:17:26.869796 825 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTU9OP6/build/release/bin/kudu as pid 6221
I20260619 10:17:26.869894 825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781864224577058-825-0/minicluster-data/master-0/wal/instance
I20260619 10:17:26.869869 6288 consensus_queue.cc:260] T 00000000000000000000000000000000 P 46f1ec1bd83846869fa4c6564aadcbac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "46f1ec1bd83846869fa4c6564aadcbac" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 36263 } }
I20260619 10:17:26.869953 6288 raft_consensus.cc:388] T 00000000000000000000000000000000 P 46f1ec1bd83846869fa4c6564aadcbac [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 10:17:26.869997 6288 raft_consensus.cc:482] T 00000000000000000000000000000000 P 46f1ec1bd83846869fa4c6564aadcbac [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 10:17:26.870042 6288 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 46f1ec1bd83846869fa4c6564aadcbac [term 0 FOLLOWER]: Advancing to term 1
I20260619 10:17:26.870096 825 external_mini_cluster.cc:1428] Setting key ba841f86882309ab7e5d34de1877d4c5
I20260619 10:17:26.870795 6288 raft_consensus.cc:504] T 00000000000000000000000000000000 P 46f1ec1bd83846869fa4c6564aadcbac [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "46f1ec1bd83846869fa4c6564aadcbac" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 36263 } }
I20260619 10:17:26.870901 6288 leader_election.cc:302] T 00000000000000000000000000000000 P 46f1ec1bd83846869fa4c6564aadcbac [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 46f1ec1bd83846869fa4c6564aadcbac; no voters:
I20260619 10:17:26.871047 6288 leader_election.cc:288] T 00000000000000000000000000000000 P 46f1ec1bd83846869fa4c6564aadcbac [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 10:17:26.871096 6293 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 46f1ec1bd83846869fa4c6564aadcbac [term 1 FOLLOWER]: Leader election won for term 1
I20260619 10:17:26.871224 6293 raft_consensus.cc:686] T 00000000000000000000000000000000 P 46f1ec1bd83846869fa4c6564aadcbac [term 1 LEADER]: Becoming Leader. State: Replica: 46f1ec1bd83846869fa4c6564aadcbac, State: Running, Role: LEADER
I20260619 10:17:26.871277 6288 sys_catalog.cc:564] T 00000000000000000000000000000000 P 46f1ec1bd83846869fa4c6564aadcbac [sys.catalog]: configured and running, proceeding with master startup.
I20260619 10:17:26.871361 6293 consensus_queue.cc:237] T 00000000000000000000000000000000 P 46f1ec1bd83846869fa4c6564aadcbac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "46f1ec1bd83846869fa4c6564aadcbac" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 36263 } }
I20260619 10:17:26.871749 6293 sys_catalog.cc:455] T 00000000000000000000000000000000 P 46f1ec1bd83846869fa4c6564aadcbac [sys.catalog]: SysCatalogTable state changed. Reason: New leader 46f1ec1bd83846869fa4c6564aadcbac. Latest consensus state: current_term: 1 leader_uuid: "46f1ec1bd83846869fa4c6564aadcbac" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "46f1ec1bd83846869fa4c6564aadcbac" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 36263 } } }
I20260619 10:17:26.871760 6295 sys_catalog.cc:455] T 00000000000000000000000000000000 P 46f1ec1bd83846869fa4c6564aadcbac [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "46f1ec1bd83846869fa4c6564aadcbac" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "46f1ec1bd83846869fa4c6564aadcbac" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 36263 } } }
I20260619 10:17:26.871927 6295 sys_catalog.cc:458] T 00000000000000000000000000000000 P 46f1ec1bd83846869fa4c6564aadcbac [sys.catalog]: This master's current role is: LEADER
I20260619 10:17:26.872112 6293 sys_catalog.cc:458] T 00000000000000000000000000000000 P 46f1ec1bd83846869fa4c6564aadcbac [sys.catalog]: This master's current role is: LEADER
I20260619 10:17:26.872503 6302 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 10:17:26.873063 6302 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 10:17:26.874401 6302 catalog_manager.cc:1347] Generated new cluster ID: 3e085e8001cf420c92d244802ca119af
I20260619 10:17:26.874459 6302 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 10:17:26.876334 6292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:26.870522 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41822 (local address 127.0.206.126:36263)
0619 10:17:26.870834 (+ 312us) server_negotiation.cc:206] Beginning negotiation
0619 10:17:26.870842 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 10:17:26.870874 (+ 32us) server_negotiation.cc:407] Connection header received
0619 10:17:26.870980 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:17:26.870987 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:17:26.871064 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:17:26.871246 (+ 182us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:17:26.871480 (+ 234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:26.873220 (+ 1740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:26.874359 (+ 1139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:26.874897 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:26.875025 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:26.875686 (+ 661us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 10:17:26.875690 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 10:17:26.875690 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 10:17:26.875861 (+ 171us) server_negotiation.cc:922] calling sasl_server_start()
0619 10:17:26.875891 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 10:17:26.875901 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 10:17:26.875979 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0619 10:17:26.876054 (+ 75us) server_negotiation.cc:299] Negotiation successful
0619 10:17:26.876128 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":240,"thread_start_us":109,"threads_started":1}
I20260619 10:17:26.877159 825 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 10:17:26.897101 6302 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 10:17:26.897818 6302 catalog_manager.cc:1504] Loading token signing keys...
I20260619 10:17:26.907027 6302 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 46f1ec1bd83846869fa4c6564aadcbac: Generated new TSK 0
I20260619 10:17:26.907202 6302 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 10:17:27.373356 6292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:26.919699 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41824 (local address 127.0.206.126:36263)
0619 10:17:26.919753 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0619 10:17:26.919756 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 10:17:26.919868 (+ 112us) server_negotiation.cc:407] Connection header received
0619 10:17:26.919954 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:17:26.919955 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:17:26.919966 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:17:26.920046 (+ 80us) server_negotiation.cc:226] Negotiated authn=JWT
0619 10:17:26.920185 (+ 139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:26.920828 (+ 643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:26.921617 (+ 789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:26.921812 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:26.921894 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:26.922084 (+ 190us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0619 10:17:27.373134 (+451050us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0619 10:17:27.373298 (+ 164us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:41824: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":32}
W20260619 10:17:27.373420 6292 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:41824: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
W20260619 10:17:27.373517 6317 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 10:17:26.919636 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:36263 (local address 127.0.0.1:41824)
0619 10:17:26.919793 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0619 10:17:26.919813 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0619 10:17:26.919886 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 10:17:26.920045 (+ 159us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 10:17:26.920045 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 10:17:26.920047 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0619 10:17:26.920109 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:26.920111 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:26.920931 (+ 820us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:26.920932 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:26.921527 (+ 595us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:26.921528 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:26.921897 (+ 369us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:26.921898 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:26.921961 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:26.921972 (+ 11us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0619 10:17:27.373268 (+451296us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0619 10:17:27.373388 (+ 120us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.0.206.126:36263: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":90,"threads_started":1}
I20260619 10:17:27.373862 825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTU9OP6/build/release/bin/kudu with pid 6221
2026-06-19T10:17:27Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (643 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (85 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (84 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781864224577058-825-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 10:17:27 dist-test-slave-6pt2 krb5kdc[6329](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 10:17:27 dist-test-slave-6pt2 krb5kdc[6329](info): set up 2 sockets
Jun 19 10:17:27 dist-test-slave-6pt2 krb5kdc[6329](info): commencing operation
krb5kdc: starting...
W20260619 10:17:28.582582 825 mini_kdc.cc:121] Time spent starting KDC: real 1.021s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781864224577058-825-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781864224577058-825-0/krb5kdc/test-user.keytab.
Jun 19 10:17:28 dist-test-slave-6pt2 krb5kdc[6329](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864248, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T10:17:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T10:17:28Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.206.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.206.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.206.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781864224577058-825-0/krb5kdc/kudu_127.0.206.126.keytab.
Entry for principal kudu/127.0.206.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781864224577058-825-0/krb5kdc/kudu_127.0.206.126.keytab.
I20260619 10:17:28.697199 825 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTU9OP6/build/release/bin/kudu
/tmp/dist-test-taskTU9OP6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781864224577058-825-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781864224577058-825-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781864224577058-825-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781864224577058-825-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.206.126:38189
--webserver_interface=127.0.206.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781864224577058-825-0/krb5kdc/kudu_127.0.206.126.keytab
--principal=kudu/127.0.206.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.206.84:36039
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.206.126:38189
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781864224577058-825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781864224577058-825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781864224577058-825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 10:17:28.774701 6342 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 10:17:28.774889 6342 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 10:17:28.774909 6342 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 10:17:28.775993 6342 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 10:17:28.776031 6342 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 10:17:28.776046 6342 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 10:17:28.776094 6342 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 10:17:28.776121 6342 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 10:17:28.777535 6342 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.206.84:36039
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781864224577058-825-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781864224577058-825-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.206.126:38189
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781864224577058-825-0/krb5kdc/kudu_127.0.206.126.keytab
--principal=kudu/127.0.206.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.206.126:38189
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781864224577058-825-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.206.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6342
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781864224577058-825-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 09:32:44 UTC on ccycloud.quasar-gbtldp.root.comops.site
build id 4455440
I20260619 10:17:28.777750 6342 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 10:17:28.777964 6342 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 10:17:28.782511 6348 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 10:17:28.782521 6350 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 10:17:28.782616 6342 server_base.cc:1034] running on GCE node
W20260619 10:17:28.782526 6347 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 10:17:28.783023 6342 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 10:17:28.783283 6342 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 10:17:28.784420 6342 hybrid_clock.cc:648] HybridClock initialized: now 1781864248784410 us; error 30 us; skew 500 ppm
Jun 19 10:17:28 dist-test-slave-6pt2 krb5kdc[6329](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864248, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 10:17:28.789311 6342 init.cc:377] Logged in from keytab as kudu/127.0.206.126@KRBTEST.COM (short username kudu)
I20260619 10:17:28.789698 6342 webserver.cc:460] Webserver started at http://127.0.206.126:43557/ using document root <none> and password file <none>
I20260619 10:17:28.789916 6342 fs_manager.cc:359] Metadata directory not provided
I20260619 10:17:28.789958 6342 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 10:17:28.790084 6342 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 10:17:28.791106 6342 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781864224577058-825-0/minicluster-data/master-0/data/instance:
uuid: "05ffabbd9e3c49c8978d98a6d6b6cbaa"
format_stamp: "Formatted at 2026-06-19 10:17:28 on dist-test-slave-6pt2"
server_key: "d313beb4de25ea142f8557002ac7727a"
server_key_iv: "c80bf3ba9f4d68e0f7feff76d205824e"
server_key_version: "encryptionkey@0"
I20260619 10:17:28.791437 6342 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781864224577058-825-0/minicluster-data/master-0/wal/instance:
uuid: "05ffabbd9e3c49c8978d98a6d6b6cbaa"
format_stamp: "Formatted at 2026-06-19 10:17:28 on dist-test-slave-6pt2"
server_key: "d313beb4de25ea142f8557002ac7727a"
server_key_iv: "c80bf3ba9f4d68e0f7feff76d205824e"
server_key_version: "encryptionkey@0"
I20260619 10:17:28.792654 6342 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 10:17:28.793334 6357 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 10:17:28.793493 6342 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 10:17:28.793583 6342 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781864224577058-825-0/minicluster-data/master-0/data,/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781864224577058-825-0/minicluster-data/master-0/wal
uuid: "05ffabbd9e3c49c8978d98a6d6b6cbaa"
format_stamp: "Formatted at 2026-06-19 10:17:28 on dist-test-slave-6pt2"
server_key: "d313beb4de25ea142f8557002ac7727a"
server_key_iv: "c80bf3ba9f4d68e0f7feff76d205824e"
server_key_version: "encryptionkey@0"
I20260619 10:17:28.793655 6342 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781864224577058-825-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781864224577058-825-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781864224577058-825-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 10:17:28.816440 6342 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 10:17:28.818106 6342 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 10:17:28.818257 6342 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 10:17:28.822458 6342 rpc_server.cc:307] RPC server started. Bound to: 127.0.206.126:38189
I20260619 10:17:28.822492 6409 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.206.126:38189 every 8 connection(s)
I20260619 10:17:28.822995 6342 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781864224577058-825-0/minicluster-data/master-0/data/info.pb
I20260619 10:17:28.823616 6410 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 10:17:28.826087 6410 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 05ffabbd9e3c49c8978d98a6d6b6cbaa: Bootstrap starting.
I20260619 10:17:28.826785 6410 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 05ffabbd9e3c49c8978d98a6d6b6cbaa: Neither blocks nor log segments found. Creating new log.
I20260619 10:17:28.827033 6410 log.cc:824] T 00000000000000000000000000000000 P 05ffabbd9e3c49c8978d98a6d6b6cbaa: Log is configured to *not* fsync() on all Append() calls
I20260619 10:17:28.827631 6410 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 05ffabbd9e3c49c8978d98a6d6b6cbaa: No bootstrap required, opened a new log
I20260619 10:17:28.829147 6410 raft_consensus.cc:348] T 00000000000000000000000000000000 P 05ffabbd9e3c49c8978d98a6d6b6cbaa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "05ffabbd9e3c49c8978d98a6d6b6cbaa" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 38189 } }
I20260619 10:17:28.829270 6410 raft_consensus.cc:374] T 00000000000000000000000000000000 P 05ffabbd9e3c49c8978d98a6d6b6cbaa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 10:17:28.829293 6410 raft_consensus.cc:729] T 00000000000000000000000000000000 P 05ffabbd9e3c49c8978d98a6d6b6cbaa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 05ffabbd9e3c49c8978d98a6d6b6cbaa, State: Initialized, Role: FOLLOWER
I20260619 10:17:28.829418 6410 consensus_queue.cc:260] T 00000000000000000000000000000000 P 05ffabbd9e3c49c8978d98a6d6b6cbaa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "05ffabbd9e3c49c8978d98a6d6b6cbaa" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 38189 } }
I20260619 10:17:28.829526 6410 raft_consensus.cc:388] T 00000000000000000000000000000000 P 05ffabbd9e3c49c8978d98a6d6b6cbaa [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 10:17:28.829550 6410 raft_consensus.cc:482] T 00000000000000000000000000000000 P 05ffabbd9e3c49c8978d98a6d6b6cbaa [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 10:17:28.829583 6410 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 05ffabbd9e3c49c8978d98a6d6b6cbaa [term 0 FOLLOWER]: Advancing to term 1
I20260619 10:17:28.830410 6410 raft_consensus.cc:504] T 00000000000000000000000000000000 P 05ffabbd9e3c49c8978d98a6d6b6cbaa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "05ffabbd9e3c49c8978d98a6d6b6cbaa" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 38189 } }
I20260619 10:17:28.830490 6410 leader_election.cc:302] T 00000000000000000000000000000000 P 05ffabbd9e3c49c8978d98a6d6b6cbaa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 05ffabbd9e3c49c8978d98a6d6b6cbaa; no voters:
I20260619 10:17:28.830662 6410 leader_election.cc:288] T 00000000000000000000000000000000 P 05ffabbd9e3c49c8978d98a6d6b6cbaa [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 10:17:28.830694 6413 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 05ffabbd9e3c49c8978d98a6d6b6cbaa [term 1 FOLLOWER]: Leader election won for term 1
I20260619 10:17:28.830852 6413 raft_consensus.cc:686] T 00000000000000000000000000000000 P 05ffabbd9e3c49c8978d98a6d6b6cbaa [term 1 LEADER]: Becoming Leader. State: Replica: 05ffabbd9e3c49c8978d98a6d6b6cbaa, State: Running, Role: LEADER
I20260619 10:17:28.830920 6410 sys_catalog.cc:564] T 00000000000000000000000000000000 P 05ffabbd9e3c49c8978d98a6d6b6cbaa [sys.catalog]: configured and running, proceeding with master startup.
I20260619 10:17:28.830972 6413 consensus_queue.cc:237] T 00000000000000000000000000000000 P 05ffabbd9e3c49c8978d98a6d6b6cbaa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "05ffabbd9e3c49c8978d98a6d6b6cbaa" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 38189 } }
I20260619 10:17:28.831341 6415 sys_catalog.cc:455] T 00000000000000000000000000000000 P 05ffabbd9e3c49c8978d98a6d6b6cbaa [sys.catalog]: SysCatalogTable state changed. Reason: New leader 05ffabbd9e3c49c8978d98a6d6b6cbaa. Latest consensus state: current_term: 1 leader_uuid: "05ffabbd9e3c49c8978d98a6d6b6cbaa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "05ffabbd9e3c49c8978d98a6d6b6cbaa" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 38189 } } }
I20260619 10:17:28.831426 825 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTU9OP6/build/release/bin/kudu as pid 6342
I20260619 10:17:28.831332 6414 sys_catalog.cc:455] T 00000000000000000000000000000000 P 05ffabbd9e3c49c8978d98a6d6b6cbaa [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "05ffabbd9e3c49c8978d98a6d6b6cbaa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "05ffabbd9e3c49c8978d98a6d6b6cbaa" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 38189 } } }
I20260619 10:17:28.831486 6415 sys_catalog.cc:458] T 00000000000000000000000000000000 P 05ffabbd9e3c49c8978d98a6d6b6cbaa [sys.catalog]: This master's current role is: LEADER
I20260619 10:17:28.831489 6414 sys_catalog.cc:458] T 00000000000000000000000000000000 P 05ffabbd9e3c49c8978d98a6d6b6cbaa [sys.catalog]: This master's current role is: LEADER
I20260619 10:17:28.831538 825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781864224577058-825-0/minicluster-data/master-0/wal/instance
I20260619 10:17:28.831698 6422 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 10:17:28.831768 825 external_mini_cluster.cc:1428] Setting key f939949ef40fc03e05af7d2a00ed5850
I20260619 10:17:28.832363 6422 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 10:17:28.834038 6422 catalog_manager.cc:1347] Generated new cluster ID: f3c9bc844ea2449a80838075fe51a415
I20260619 10:17:28.834087 6422 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Jun 19 10:17:28 dist-test-slave-6pt2 krb5kdc[6329](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864248, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.206.126@KRBTEST.COM
I20260619 10:17:28.843304 6431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:28.832205 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37498 (local address 127.0.206.126:38189)
0619 10:17:28.832359 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0619 10:17:28.832368 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 10:17:28.832621 (+ 253us) server_negotiation.cc:407] Connection header received
0619 10:17:28.832968 (+ 347us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:17:28.832976 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:17:28.833034 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:17:28.833201 (+ 167us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:17:28.833963 (+ 762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:28.835396 (+ 1433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:28.836372 (+ 976us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:28.837032 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:28.837137 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:28.839877 (+ 2740us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 10:17:28.839881 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 10:17:28.839882 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 10:17:28.839892 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0619 10:17:28.841721 (+ 1829us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:17:28.842102 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:17:28.842106 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:17:28.842106 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:17:28.842204 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:17:28.842398 (+ 194us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:17:28.842399 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:17:28.842399 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:17:28.842709 (+ 310us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 10:17:28.842849 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0619 10:17:28.842969 (+ 120us) server_negotiation.cc:299] Negotiation successful
0619 10:17:28.843092 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":112,"thread_start_us":65,"threads_started":1}
I20260619 10:17:28.847513 6422 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 10:17:28.848150 6422 catalog_manager.cc:1504] Loading token signing keys...
I20260619 10:17:28.851485 6422 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 05ffabbd9e3c49c8978d98a6d6b6cbaa: Generated new TSK 0
I20260619 10:17:28.851625 6422 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.0.206.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.206.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.206.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781864224577058-825-0/krb5kdc/kudu_127.0.206.65.keytab.
Entry for principal kudu/127.0.206.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781864224577058-825-0/krb5kdc/kudu_127.0.206.65.keytab.
I20260619 10:17:28.875083 825 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTU9OP6/build/release/bin/kudu
/tmp/dist-test-taskTU9OP6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781864224577058-825-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781864224577058-825-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781864224577058-825-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781864224577058-825-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.206.65:0
--local_ip_for_outbound_sockets=127.0.206.65
--webserver_interface=127.0.206.65
--webserver_port=0
--tserver_master_addrs=127.0.206.126:38189
--keytab_file=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781864224577058-825-0/krb5kdc/kudu_127.0.206.65.keytab
--principal=kudu/127.0.206.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.206.84:36039
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781864224577058-825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781864224577058-825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781864224577058-825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 10:17:28.952363 6436 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 10:17:28.952580 6436 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 10:17:28.952615 6436 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 10:17:28.953779 6436 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 10:17:28.953847 6436 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 10:17:28.953891 6436 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.206.65
I20260619 10:17:28.955343 6436 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.206.84:36039
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781864224577058-825-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781864224577058-825-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781864224577058-825-0/krb5kdc/kudu_127.0.206.65.keytab
--principal=kudu/127.0.206.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.206.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781864224577058-825-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.206.65
--webserver_port=0
--tserver_master_addrs=127.0.206.126:38189
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6436
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.206.65
--log_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781864224577058-825-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 09:32:44 UTC on ccycloud.quasar-gbtldp.root.comops.site
build id 4455440
I20260619 10:17:28.955580 6436 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 10:17:28.955789 6436 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 10:17:28.960439 6444 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 10:17:28.960482 6441 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 10:17:28.960615 6436 server_base.cc:1034] running on GCE node
W20260619 10:17:28.960680 6442 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 10:17:28.960916 6436 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 10:17:28.961139 6436 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 10:17:28.962293 6436 hybrid_clock.cc:648] HybridClock initialized: now 1781864248962285 us; error 35 us; skew 500 ppm
Jun 19 10:17:28 dist-test-slave-6pt2 krb5kdc[6329](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864248, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 10:17:28.964704 6436 init.cc:377] Logged in from keytab as kudu/127.0.206.65@KRBTEST.COM (short username kudu)
I20260619 10:17:28.965118 6436 webserver.cc:460] Webserver started at http://127.0.206.65:39481/ using document root <none> and password file <none>
I20260619 10:17:28.965332 6436 fs_manager.cc:359] Metadata directory not provided
I20260619 10:17:28.965401 6436 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 10:17:28.965533 6436 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 10:17:28.966480 6436 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781864224577058-825-0/minicluster-data/ts-0/data/instance:
uuid: "e150df0d6bbd4d04b823994c275c34b5"
format_stamp: "Formatted at 2026-06-19 10:17:28 on dist-test-slave-6pt2"
server_key: "0d4231b1e0b0c5f6485e220396f7d627"
server_key_iv: "052b29ba1dcf75c26db7d774d25aeaab"
server_key_version: "encryptionkey@0"
I20260619 10:17:28.966830 6436 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781864224577058-825-0/minicluster-data/ts-0/wal/instance:
uuid: "e150df0d6bbd4d04b823994c275c34b5"
format_stamp: "Formatted at 2026-06-19 10:17:28 on dist-test-slave-6pt2"
server_key: "0d4231b1e0b0c5f6485e220396f7d627"
server_key_iv: "052b29ba1dcf75c26db7d774d25aeaab"
server_key_version: "encryptionkey@0"
I20260619 10:17:28.968034 6436 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 10:17:28.968739 6451 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 10:17:28.968890 6436 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 10:17:28.968998 6436 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781864224577058-825-0/minicluster-data/ts-0/data,/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781864224577058-825-0/minicluster-data/ts-0/wal
uuid: "e150df0d6bbd4d04b823994c275c34b5"
format_stamp: "Formatted at 2026-06-19 10:17:28 on dist-test-slave-6pt2"
server_key: "0d4231b1e0b0c5f6485e220396f7d627"
server_key_iv: "052b29ba1dcf75c26db7d774d25aeaab"
server_key_version: "encryptionkey@0"
I20260619 10:17:28.969064 6436 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781864224577058-825-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781864224577058-825-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781864224577058-825-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 10:17:28.983309 6436 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 10:17:28.984900 6436 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 10:17:28.985047 6436 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 10:17:28.985260 6436 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 10:17:28.985544 6436 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 10:17:28.985594 6436 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 10:17:28.985623 6436 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 10:17:28.985651 6436 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 10:17:28.991168 6436 rpc_server.cc:307] RPC server started. Bound to: 127.0.206.65:45483
I20260619 10:17:28.991227 6564 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.206.65:45483 every 8 connection(s)
I20260619 10:17:28.991526 6436 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781864224577058-825-0/minicluster-data/ts-0/data/info.pb
Jun 19 10:17:28 dist-test-slave-6pt2 krb5kdc[6329](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864248, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.65@KRBTEST.COM for kudu/127.0.206.126@KRBTEST.COM
I20260619 10:17:28.999543 825 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTU9OP6/build/release/bin/kudu as pid 6436
I20260619 10:17:28.999644 825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781864224577058-825-0/minicluster-data/ts-0/wal/instance
I20260619 10:17:28.999830 825 external_mini_cluster.cc:1428] Setting key 27681b9bca9aefdc62740829bcddfc0d
I20260619 10:17:29.000144 6431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:28.992205 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.65:58855 (local address 127.0.206.126:38189)
0619 10:17:28.992246 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0619 10:17:28.992247 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 10:17:28.992434 (+ 187us) server_negotiation.cc:407] Connection header received
0619 10:17:28.993261 (+ 827us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:17:28.993262 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:17:28.993274 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:17:28.993320 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:17:28.993831 (+ 511us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:28.994305 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:28.995303 (+ 998us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:28.995498 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:28.995544 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:28.997479 (+ 1935us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 10:17:28.997481 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 10:17:28.997481 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 10:17:28.997487 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 10:17:28.998911 (+ 1424us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:17:28.999231 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:17:28.999231 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:17:28.999232 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:17:28.999278 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:17:28.999491 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:17:28.999492 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:17:28.999493 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:17:28.999652 (+ 159us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 10:17:28.999721 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0619 10:17:28.999974 (+ 253us) server_negotiation.cc:299] Negotiation successful
0619 10:17:29.000057 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260619 10:17:29.000296 6567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:28.992177 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:38189 (local address 127.0.206.65:58855)
0619 10:17:28.992325 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0619 10:17:28.992355 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0619 10:17:28.993181 (+ 826us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 10:17:28.993377 (+ 196us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 10:17:28.993380 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 10:17:28.993599 (+ 219us) client_negotiation.cc:189] Negotiated authn=SASL
0619 10:17:28.993764 (+ 165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:28.993768 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:28.994376 (+ 608us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:28.994377 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:28.995209 (+ 832us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:28.995211 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:28.995559 (+ 348us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:28.995560 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:28.995654 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:28.996034 (+ 380us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 10:17:28.996057 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0619 10:17:28.997393 (+ 1336us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 10:17:28.998979 (+ 1586us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 10:17:28.998984 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 10:17:28.998985 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 10:17:28.999175 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 10:17:28.999343 (+ 168us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 10:17:28.999343 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 10:17:28.999344 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 10:17:28.999431 (+ 87us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 10:17:28.999711 (+ 280us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 10:17:28.999716 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 10:17:28.999832 (+ 116us) client_negotiation.cc:769] Sending connection context
0619 10:17:28.999936 (+ 104us) client_negotiation.cc:240] Negotiation successful
0619 10:17:29.000078 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":88,"threads_started":1}
I20260619 10:17:29.000725 6565 heartbeater.cc:347] Connected to a master server at 127.0.206.126:38189
I20260619 10:17:29.000806 6565 heartbeater.cc:464] Registering TS with master...
I20260619 10:17:29.000976 6565 heartbeater.cc:511] Master 127.0.206.126:38189 requested a full tablet report, sending...
I20260619 10:17:29.001413 6374 ts_manager.cc:194] Registered new tserver with Master: e150df0d6bbd4d04b823994c275c34b5 (127.0.206.65:45483)
I20260619 10:17:29.002009 6374 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.206.65@KRBTEST.COM'} at 127.0.206.65:58855
WARNING: no policy specified for kudu/127.0.206.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.206.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.206.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781864224577058-825-0/krb5kdc/kudu_127.0.206.66.keytab.
Entry for principal kudu/127.0.206.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781864224577058-825-0/krb5kdc/kudu_127.0.206.66.keytab.
I20260619 10:17:29.022349 825 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTU9OP6/build/release/bin/kudu
/tmp/dist-test-taskTU9OP6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781864224577058-825-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781864224577058-825-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781864224577058-825-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781864224577058-825-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.206.66:0
--local_ip_for_outbound_sockets=127.0.206.66
--webserver_interface=127.0.206.66
--webserver_port=0
--tserver_master_addrs=127.0.206.126:38189
--keytab_file=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781864224577058-825-0/krb5kdc/kudu_127.0.206.66.keytab
--principal=kudu/127.0.206.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.206.84:36039
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781864224577058-825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781864224577058-825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781864224577058-825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 10:17:29.100572 6570 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 10:17:29.100792 6570 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 10:17:29.100829 6570 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 10:17:29.101965 6570 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 10:17:29.102025 6570 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 10:17:29.102068 6570 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.206.66
I20260619 10:17:29.103508 6570 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.206.84:36039
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781864224577058-825-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781864224577058-825-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781864224577058-825-0/krb5kdc/kudu_127.0.206.66.keytab
--principal=kudu/127.0.206.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.206.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781864224577058-825-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.206.66
--webserver_port=0
--tserver_master_addrs=127.0.206.126:38189
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6570
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.206.66
--log_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781864224577058-825-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 09:32:44 UTC on ccycloud.quasar-gbtldp.root.comops.site
build id 4455440
I20260619 10:17:29.103765 6570 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 10:17:29.103971 6570 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 10:17:29.108747 6578 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 10:17:29.108875 6570 server_base.cc:1034] running on GCE node
W20260619 10:17:29.108771 6575 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 10:17:29.108783 6576 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 10:17:29.109151 6570 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 10:17:29.109382 6570 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 10:17:29.110533 6570 hybrid_clock.cc:648] HybridClock initialized: now 1781864249110523 us; error 28 us; skew 500 ppm
Jun 19 10:17:29 dist-test-slave-6pt2 krb5kdc[6329](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864249, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 10:17:29.112816 6570 init.cc:377] Logged in from keytab as kudu/127.0.206.66@KRBTEST.COM (short username kudu)
I20260619 10:17:29.113204 6570 webserver.cc:460] Webserver started at http://127.0.206.66:37185/ using document root <none> and password file <none>
I20260619 10:17:29.113409 6570 fs_manager.cc:359] Metadata directory not provided
I20260619 10:17:29.113472 6570 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 10:17:29.113610 6570 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 10:17:29.114563 6570 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781864224577058-825-0/minicluster-data/ts-1/data/instance:
uuid: "e5e80a0388114fddae73a8986b211441"
format_stamp: "Formatted at 2026-06-19 10:17:29 on dist-test-slave-6pt2"
server_key: "665e202549990d8490688a58ff8069c6"
server_key_iv: "4c0fc292b0a67bed91db3d3b73558b92"
server_key_version: "encryptionkey@0"
I20260619 10:17:29.114946 6570 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781864224577058-825-0/minicluster-data/ts-1/wal/instance:
uuid: "e5e80a0388114fddae73a8986b211441"
format_stamp: "Formatted at 2026-06-19 10:17:29 on dist-test-slave-6pt2"
server_key: "665e202549990d8490688a58ff8069c6"
server_key_iv: "4c0fc292b0a67bed91db3d3b73558b92"
server_key_version: "encryptionkey@0"
I20260619 10:17:29.116161 6570 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 10:17:29.116767 6585 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 10:17:29.116919 6570 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 10:17:29.117004 6570 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781864224577058-825-0/minicluster-data/ts-1/data,/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781864224577058-825-0/minicluster-data/ts-1/wal
uuid: "e5e80a0388114fddae73a8986b211441"
format_stamp: "Formatted at 2026-06-19 10:17:29 on dist-test-slave-6pt2"
server_key: "665e202549990d8490688a58ff8069c6"
server_key_iv: "4c0fc292b0a67bed91db3d3b73558b92"
server_key_version: "encryptionkey@0"
I20260619 10:17:29.117074 6570 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781864224577058-825-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781864224577058-825-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781864224577058-825-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 10:17:29.128921 6570 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 10:17:29.130669 6570 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 10:17:29.130822 6570 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 10:17:29.131038 6570 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 10:17:29.131320 6570 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 10:17:29.131371 6570 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 10:17:29.131412 6570 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 10:17:29.131441 6570 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 10:17:29.137020 6570 rpc_server.cc:307] RPC server started. Bound to: 127.0.206.66:39007
I20260619 10:17:29.137071 6698 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.206.66:39007 every 8 connection(s)
I20260619 10:17:29.137353 6570 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781864224577058-825-0/minicluster-data/ts-1/data/info.pb
Jun 19 10:17:29 dist-test-slave-6pt2 krb5kdc[6329](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864249, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.66@KRBTEST.COM for kudu/127.0.206.126@KRBTEST.COM
I20260619 10:17:29.145545 6431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:29.138005 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.66:36451 (local address 127.0.206.126:38189)
0619 10:17:29.138072 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0619 10:17:29.138074 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 10:17:29.138287 (+ 213us) server_negotiation.cc:407] Connection header received
0619 10:17:29.139201 (+ 914us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:17:29.139202 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:17:29.139211 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:17:29.139261 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:17:29.139767 (+ 506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:29.140234 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:29.141183 (+ 949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:29.141365 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:29.141413 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:29.143381 (+ 1968us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 10:17:29.143382 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 10:17:29.143383 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 10:17:29.143389 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 10:17:29.144528 (+ 1139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:17:29.144857 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:17:29.144857 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:17:29.144857 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:17:29.144902 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:17:29.145089 (+ 187us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:17:29.145089 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:17:29.145089 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:17:29.145178 (+ 89us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 10:17:29.145223 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0619 10:17:29.145423 (+ 200us) server_negotiation.cc:299] Negotiation successful
0619 10:17:29.145485 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260619 10:17:29.145737 6701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:29.137997 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:38189 (local address 127.0.206.66:36451)
0619 10:17:29.138156 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0619 10:17:29.138187 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0619 10:17:29.139118 (+ 931us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 10:17:29.139310 (+ 192us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 10:17:29.139313 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 10:17:29.139535 (+ 222us) client_negotiation.cc:189] Negotiated authn=SASL
0619 10:17:29.139703 (+ 168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:29.139707 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:29.140300 (+ 593us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:29.140301 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:29.141115 (+ 814us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:29.141117 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:29.141429 (+ 312us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:29.141430 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:29.141524 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:29.141985 (+ 461us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 10:17:29.141993 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 10:17:29.143300 (+ 1307us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 10:17:29.144591 (+ 1291us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 10:17:29.144596 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 10:17:29.144598 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 10:17:29.144796 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 10:17:29.144952 (+ 156us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 10:17:29.144953 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 10:17:29.144953 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 10:17:29.145039 (+ 86us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 10:17:29.145225 (+ 186us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 10:17:29.145229 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 10:17:29.145326 (+ 97us) client_negotiation.cc:769] Sending connection context
0619 10:17:29.145408 (+ 82us) client_negotiation.cc:240] Negotiation successful
0619 10:17:29.145515 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":89,"threads_started":1}
I20260619 10:17:29.146236 6699 heartbeater.cc:347] Connected to a master server at 127.0.206.126:38189
I20260619 10:17:29.146337 6699 heartbeater.cc:464] Registering TS with master...
I20260619 10:17:29.146409 825 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTU9OP6/build/release/bin/kudu as pid 6570
I20260619 10:17:29.146503 825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781864224577058-825-0/minicluster-data/ts-1/wal/instance
I20260619 10:17:29.146528 6699 heartbeater.cc:511] Master 127.0.206.126:38189 requested a full tablet report, sending...
I20260619 10:17:29.146732 825 external_mini_cluster.cc:1428] Setting key 4c740a0f63b327aeba42a072d5aa43ec
I20260619 10:17:29.146921 6374 ts_manager.cc:194] Registered new tserver with Master: e5e80a0388114fddae73a8986b211441 (127.0.206.66:39007)
I20260619 10:17:29.147339 6374 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.206.66@KRBTEST.COM'} at 127.0.206.66:36451
WARNING: no policy specified for kudu/127.0.206.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.206.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.206.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781864224577058-825-0/krb5kdc/kudu_127.0.206.67.keytab.
Entry for principal kudu/127.0.206.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781864224577058-825-0/krb5kdc/kudu_127.0.206.67.keytab.
I20260619 10:17:29.167227 825 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTU9OP6/build/release/bin/kudu
/tmp/dist-test-taskTU9OP6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781864224577058-825-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781864224577058-825-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781864224577058-825-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781864224577058-825-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.206.67:0
--local_ip_for_outbound_sockets=127.0.206.67
--webserver_interface=127.0.206.67
--webserver_port=0
--tserver_master_addrs=127.0.206.126:38189
--keytab_file=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781864224577058-825-0/krb5kdc/kudu_127.0.206.67.keytab
--principal=kudu/127.0.206.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.206.84:36039
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781864224577058-825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781864224577058-825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781864224577058-825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 10:17:29.243531 6704 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 10:17:29.243755 6704 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 10:17:29.243790 6704 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 10:17:29.244887 6704 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 10:17:29.244946 6704 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 10:17:29.244989 6704 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.206.67
I20260619 10:17:29.246366 6704 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.206.84:36039
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781864224577058-825-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781864224577058-825-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781864224577058-825-0/krb5kdc/kudu_127.0.206.67.keytab
--principal=kudu/127.0.206.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.206.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781864224577058-825-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.206.67
--webserver_port=0
--tserver_master_addrs=127.0.206.126:38189
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6704
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.206.67
--log_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781864224577058-825-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 09:32:44 UTC on ccycloud.quasar-gbtldp.root.comops.site
build id 4455440
I20260619 10:17:29.246594 6704 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 10:17:29.246809 6704 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 10:17:29.251262 6710 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 10:17:29.251307 6712 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 10:17:29.251364 6709 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 10:17:29.251541 6704 server_base.cc:1034] running on GCE node
I20260619 10:17:29.251744 6704 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 10:17:29.251993 6704 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 10:17:29.253144 6704 hybrid_clock.cc:648] HybridClock initialized: now 1781864249253139 us; error 41 us; skew 500 ppm
Jun 19 10:17:29 dist-test-slave-6pt2 krb5kdc[6329](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864249, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 10:17:29.255541 6704 init.cc:377] Logged in from keytab as kudu/127.0.206.67@KRBTEST.COM (short username kudu)
I20260619 10:17:29.255949 6704 webserver.cc:460] Webserver started at http://127.0.206.67:40343/ using document root <none> and password file <none>
I20260619 10:17:29.256107 6704 fs_manager.cc:359] Metadata directory not provided
I20260619 10:17:29.256143 6704 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 10:17:29.256223 6704 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 10:17:29.257113 6704 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781864224577058-825-0/minicluster-data/ts-2/data/instance:
uuid: "1421b113167246278ec13f09c3be5550"
format_stamp: "Formatted at 2026-06-19 10:17:29 on dist-test-slave-6pt2"
server_key: "b5aee0f0c8243de0b37d3388b9f51880"
server_key_iv: "c524f776ff2f10de99cb864d69ee2299"
server_key_version: "encryptionkey@0"
I20260619 10:17:29.257449 6704 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781864224577058-825-0/minicluster-data/ts-2/wal/instance:
uuid: "1421b113167246278ec13f09c3be5550"
format_stamp: "Formatted at 2026-06-19 10:17:29 on dist-test-slave-6pt2"
server_key: "b5aee0f0c8243de0b37d3388b9f51880"
server_key_iv: "c524f776ff2f10de99cb864d69ee2299"
server_key_version: "encryptionkey@0"
I20260619 10:17:29.258594 6704 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 10:17:29.259299 6719 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 10:17:29.259443 6704 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 10:17:29.259518 6704 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781864224577058-825-0/minicluster-data/ts-2/data,/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781864224577058-825-0/minicluster-data/ts-2/wal
uuid: "1421b113167246278ec13f09c3be5550"
format_stamp: "Formatted at 2026-06-19 10:17:29 on dist-test-slave-6pt2"
server_key: "b5aee0f0c8243de0b37d3388b9f51880"
server_key_iv: "c524f776ff2f10de99cb864d69ee2299"
server_key_version: "encryptionkey@0"
I20260619 10:17:29.259567 6704 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781864224577058-825-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781864224577058-825-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781864224577058-825-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 10:17:29.269305 6704 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 10:17:29.271353 6704 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 10:17:29.271461 6704 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 10:17:29.271656 6704 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 10:17:29.271924 6704 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 10:17:29.271955 6704 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 10:17:29.271975 6704 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 10:17:29.271987 6704 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 10:17:29.277344 6704 rpc_server.cc:307] RPC server started. Bound to: 127.0.206.67:45389
I20260619 10:17:29.277395 6832 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.206.67:45389 every 8 connection(s)
I20260619 10:17:29.277673 6704 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781864224577058-825-0/minicluster-data/ts-2/data/info.pb
I20260619 10:17:29.281406 825 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTU9OP6/build/release/bin/kudu as pid 6704
I20260619 10:17:29.281504 825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781864224577058-825-0/minicluster-data/ts-2/wal/instance
I20260619 10:17:29.281678 825 external_mini_cluster.cc:1428] Setting key 9f84cadae20e17ca995719a293df32aa
Jun 19 10:17:29 dist-test-slave-6pt2 krb5kdc[6329](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864249, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.67@KRBTEST.COM for kudu/127.0.206.126@KRBTEST.COM
I20260619 10:17:29.286002 6431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:29.278292 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.67:58091 (local address 127.0.206.126:38189)
0619 10:17:29.278333 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0619 10:17:29.278335 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 10:17:29.278501 (+ 166us) server_negotiation.cc:407] Connection header received
0619 10:17:29.279413 (+ 912us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:17:29.279414 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:17:29.279423 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:17:29.279477 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:17:29.279986 (+ 509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:29.280447 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:29.281372 (+ 925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:29.281677 (+ 305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:29.281759 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:29.283810 (+ 2051us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 10:17:29.283811 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 10:17:29.283812 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 10:17:29.283827 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0619 10:17:29.285065 (+ 1238us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:17:29.285360 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:17:29.285360 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:17:29.285361 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:17:29.285405 (+ 44us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:17:29.285580 (+ 175us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:17:29.285581 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:17:29.285581 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:17:29.285666 (+ 85us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 10:17:29.285716 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0619 10:17:29.285895 (+ 179us) server_negotiation.cc:299] Negotiation successful
0619 10:17:29.285945 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260619 10:17:29.286222 6835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:29.278242 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:38189 (local address 127.0.206.67:58091)
0619 10:17:29.278378 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0619 10:17:29.278414 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0619 10:17:29.279321 (+ 907us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 10:17:29.279534 (+ 213us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 10:17:29.279537 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 10:17:29.279747 (+ 210us) client_negotiation.cc:189] Negotiated authn=SASL
0619 10:17:29.279908 (+ 161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:29.279921 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:29.280513 (+ 592us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:29.280514 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:29.281245 (+ 731us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:29.281246 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:29.281764 (+ 518us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:29.281765 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:29.281933 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:29.282415 (+ 482us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 10:17:29.282424 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0619 10:17:29.283735 (+ 1311us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 10:17:29.285125 (+ 1390us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 10:17:29.285128 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 10:17:29.285130 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 10:17:29.285306 (+ 176us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 10:17:29.285454 (+ 148us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 10:17:29.285455 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 10:17:29.285455 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 10:17:29.285535 (+ 80us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 10:17:29.285718 (+ 183us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 10:17:29.285721 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 10:17:29.285813 (+ 92us) client_negotiation.cc:769] Sending connection context
0619 10:17:29.285884 (+ 71us) client_negotiation.cc:240] Negotiation successful
0619 10:17:29.285990 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":76,"threads_started":1}
I20260619 10:17:29.286703 6833 heartbeater.cc:347] Connected to a master server at 127.0.206.126:38189
I20260619 10:17:29.286798 6833 heartbeater.cc:464] Registering TS with master...
I20260619 10:17:29.286958 6833 heartbeater.cc:511] Master 127.0.206.126:38189 requested a full tablet report, sending...
I20260619 10:17:29.287298 6374 ts_manager.cc:194] Registered new tserver with Master: 1421b113167246278ec13f09c3be5550 (127.0.206.67:45389)
I20260619 10:17:29.287635 6374 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.206.67@KRBTEST.COM'} at 127.0.206.67:58091
I20260619 10:17:29.293082 825 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781864224577058-825-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781864224577058-825-0/krb5kdc/test-admin.keytab.
Jun 19 10:17:29 dist-test-slave-6pt2 krb5kdc[6329](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864249, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 10:17:29.306434 825 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260619 10:17:29.307880 825 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781864224577058-825-0/krb5kdc/krb5cc' of type FILE
W20260619 10:17:29.307900 6840 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 10:17:29.307185 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41658 (local address 127.0.206.65:45483)
0619 10:17:29.307330 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0619 10:17:29.307335 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 10:17:29.307346 (+ 11us) server_negotiation.cc:407] Connection header received
0619 10:17:29.307375 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:17:29.307377 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:17:29.307390 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:17:29.307455 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:17:29.307815 (+ 360us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41658: BlockingRecv error: recv got EOF from 127.0.0.1:41658 (error 108)
Metrics: {"server-negotiator.queue_time_us":112,"thread_start_us":82,"threads_started":1}
Jun 19 10:17:29 dist-test-slave-6pt2 krb5kdc[6329](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864249, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 10:17:29.308820 825 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Jun 19 10:17:29 dist-test-slave-6pt2 krb5kdc[6329](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864249, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 10:17:29.309640 825 init.cc:295] Successfully reacquired a new kerberos TGT
Jun 19 10:17:29 dist-test-slave-6pt2 krb5kdc[6329](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864249, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.206.65@KRBTEST.COM
I20260619 10:17:29.316756 6840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:29.310114 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41668 (local address 127.0.206.65:45483)
0619 10:17:29.310158 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0619 10:17:29.310160 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 10:17:29.310190 (+ 30us) server_negotiation.cc:407] Connection header received
0619 10:17:29.310236 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:17:29.310237 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:17:29.310245 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:17:29.310288 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:17:29.310702 (+ 414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:29.311407 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:29.312079 (+ 672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:29.312366 (+ 287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:29.312416 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:29.314336 (+ 1920us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 10:17:29.314342 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 10:17:29.314342 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 10:17:29.314348 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 10:17:29.315824 (+ 1476us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:17:29.316112 (+ 288us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:17:29.316114 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:17:29.316114 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:17:29.316162 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:17:29.316294 (+ 132us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:17:29.316294 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:17:29.316294 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:17:29.316393 (+ 99us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 10:17:29.316483 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0619 10:17:29.316537 (+ 54us) server_negotiation.cc:299] Negotiation successful
0619 10:17:29.316593 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
W20260619 10:17:29.318185 6840 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 10:17:29.317755 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41684 (local address 127.0.206.65:45483)
0619 10:17:29.317784 (+ 29us) server_negotiation.cc:206] Beginning negotiation
0619 10:17:29.317786 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 10:17:29.317815 (+ 29us) server_negotiation.cc:407] Connection header received
0619 10:17:29.317862 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:17:29.317863 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:17:29.317869 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:17:29.317911 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:17:29.318133 (+ 222us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41684: BlockingRecv error: recv got EOF from 127.0.0.1:41684 (error 108)
Metrics: {"server-negotiator.queue_time_us":14}
W20260619 10:17:29.318374 825 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781864224577058-825-0/krb5kdc/krb5cc' of type FILE
Jun 19 10:17:29 dist-test-slave-6pt2 krb5kdc[6329](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864249, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 10:17:29.319252 825 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Jun 19 10:17:29 dist-test-slave-6pt2 krb5kdc[6329](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864249, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 10:17:29.320109 825 init.cc:295] Successfully reacquired a new kerberos TGT
Jun 19 10:17:29 dist-test-slave-6pt2 krb5kdc[6329](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864249, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.206.65@KRBTEST.COM
I20260619 10:17:29.325947 6840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:29.320532 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41700 (local address 127.0.206.65:45483)
0619 10:17:29.320569 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0619 10:17:29.320571 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 10:17:29.320607 (+ 36us) server_negotiation.cc:407] Connection header received
0619 10:17:29.320696 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:17:29.320697 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:17:29.320705 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:17:29.320758 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:17:29.321120 (+ 362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:29.321564 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:29.322172 (+ 608us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:29.322349 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:29.322396 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:29.324204 (+ 1808us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 10:17:29.324205 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 10:17:29.324205 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 10:17:29.324211 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 10:17:29.325224 (+ 1013us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:17:29.325492 (+ 268us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:17:29.325493 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:17:29.325493 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:17:29.325535 (+ 42us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:17:29.325665 (+ 130us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:17:29.325665 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:17:29.325665 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:17:29.325730 (+ 65us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 10:17:29.325787 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0619 10:17:29.325844 (+ 57us) server_negotiation.cc:299] Negotiation successful
0619 10:17:29.325895 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
W20260619 10:17:29.327351 6840 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 10:17:29.326828 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41704 (local address 127.0.206.65:45483)
0619 10:17:29.326866 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0619 10:17:29.326868 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 10:17:29.326921 (+ 53us) server_negotiation.cc:407] Connection header received
0619 10:17:29.326985 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:17:29.326986 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:17:29.326994 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:17:29.327046 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:17:29.327308 (+ 262us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41704: BlockingRecv error: recv got EOF from 127.0.0.1:41704 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
W20260619 10:17:29.327498 825 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781864224577058-825-0/krb5kdc/krb5cc' of type FILE
Jun 19 10:17:29 dist-test-slave-6pt2 krb5kdc[6329](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864249, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 10:17:29.328329 825 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Jun 19 10:17:29 dist-test-slave-6pt2 krb5kdc[6329](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864249, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 10:17:29.329106 825 init.cc:295] Successfully reacquired a new kerberos TGT
Jun 19 10:17:29 dist-test-slave-6pt2 krb5kdc[6329](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864249, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.206.65@KRBTEST.COM
I20260619 10:17:29.335145 6840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:29.329550 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41710 (local address 127.0.206.65:45483)
0619 10:17:29.329595 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0619 10:17:29.329597 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 10:17:29.329640 (+ 43us) server_negotiation.cc:407] Connection header received
0619 10:17:29.329676 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:17:29.329676 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:17:29.329683 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:17:29.329723 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:17:29.330017 (+ 294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:29.330441 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:29.331191 (+ 750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:29.331395 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:29.331467 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:29.333214 (+ 1747us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 10:17:29.333215 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 10:17:29.333215 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 10:17:29.333221 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 10:17:29.334280 (+ 1059us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:17:29.334552 (+ 272us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:17:29.334553 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:17:29.334553 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:17:29.334625 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:17:29.334775 (+ 150us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:17:29.334775 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:17:29.334776 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:17:29.334850 (+ 74us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 10:17:29.334910 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0619 10:17:29.335005 (+ 95us) server_negotiation.cc:299] Negotiation successful
0619 10:17:29.335051 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260619 10:17:29.335729 825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTU9OP6/build/release/bin/kudu with pid 6436
I20260619 10:17:29.341007 825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTU9OP6/build/release/bin/kudu with pid 6570
I20260619 10:17:29.345363 825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTU9OP6/build/release/bin/kudu with pid 6704
I20260619 10:17:29.349500 825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTU9OP6/build/release/bin/kudu with pid 6342
2026-06-19T10:17:29Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (1807 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781864224577058-825-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 10:17:29 dist-test-slave-6pt2 krb5kdc[6853](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 10:17:29 dist-test-slave-6pt2 krb5kdc[6853](info): set up 2 sockets
Jun 19 10:17:29 dist-test-slave-6pt2 krb5kdc[6853](info): commencing operation
krb5kdc: starting...
W20260619 10:17:30.374784 825 mini_kdc.cc:121] Time spent starting KDC: real 1.006s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781864224577058-825-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781864224577058-825-0/krb5kdc/test-user.keytab.
Jun 19 10:17:30 dist-test-slave-6pt2 krb5kdc[6853](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864250, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T10:17:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T10:17:30Z Disabled control of system clock
WARNING: no policy specified for oryx/127.0.206.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.0.206.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.0.206.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781864224577058-825-0/krb5kdc/oryx_127.0.206.126.keytab.
Entry for principal oryx/127.0.206.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781864224577058-825-0/krb5kdc/oryx_127.0.206.126.keytab.
I20260619 10:17:30.483379 825 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTU9OP6/build/release/bin/kudu
/tmp/dist-test-taskTU9OP6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781864224577058-825-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781864224577058-825-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781864224577058-825-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781864224577058-825-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.206.126:38663
--webserver_interface=127.0.206.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781864224577058-825-0/krb5kdc/oryx_127.0.206.126.keytab
--principal=oryx/127.0.206.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.206.84:33273
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.206.126:38663
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781864224577058-825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781864224577058-825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781864224577058-825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 10:17:30.560118 6866 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 10:17:30.560336 6866 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 10:17:30.560369 6866 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 10:17:30.561463 6866 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 10:17:30.561523 6866 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 10:17:30.561545 6866 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 10:17:30.561564 6866 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 10:17:30.561594 6866 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 10:17:30.563059 6866 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.206.84:33273
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781864224577058-825-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781864224577058-825-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.206.126:38663
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781864224577058-825-0/krb5kdc/oryx_127.0.206.126.keytab
--principal=oryx/127.0.206.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.206.126:38663
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781864224577058-825-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.206.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6866
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781864224577058-825-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 09:32:44 UTC on ccycloud.quasar-gbtldp.root.comops.site
build id 4455440
I20260619 10:17:30.563309 6866 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 10:17:30.563522 6866 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 10:17:30.568048 6874 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 10:17:30.568091 6872 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 10:17:30.568243 6866 server_base.cc:1034] running on GCE node
W20260619 10:17:30.568105 6871 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 10:17:30.568521 6866 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 10:17:30.568809 6866 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 10:17:30.569967 6866 hybrid_clock.cc:648] HybridClock initialized: now 1781864250569949 us; error 30 us; skew 500 ppm
Jun 19 10:17:30 dist-test-slave-6pt2 krb5kdc[6853](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864250, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.206.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 10:17:30.572242 6866 init.cc:377] Logged in from keytab as oryx/127.0.206.126@KRBTEST.COM (short username oryx)
I20260619 10:17:30.572654 6866 webserver.cc:460] Webserver started at http://127.0.206.126:46377/ using document root <none> and password file <none>
I20260619 10:17:30.572870 6866 fs_manager.cc:359] Metadata directory not provided
I20260619 10:17:30.572938 6866 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 10:17:30.573074 6866 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 10:17:30.574035 6866 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781864224577058-825-0/minicluster-data/master-0/data/instance:
uuid: "bd146d3a022940a68274ddac0993c2e3"
format_stamp: "Formatted at 2026-06-19 10:17:30 on dist-test-slave-6pt2"
server_key: "5cab5e5c4ea69e1611a759f23ca4e18d"
server_key_iv: "cbccc19b7c3691889ddb4071ae9706ad"
server_key_version: "encryptionkey@0"
I20260619 10:17:30.574360 6866 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781864224577058-825-0/minicluster-data/master-0/wal/instance:
uuid: "bd146d3a022940a68274ddac0993c2e3"
format_stamp: "Formatted at 2026-06-19 10:17:30 on dist-test-slave-6pt2"
server_key: "5cab5e5c4ea69e1611a759f23ca4e18d"
server_key_iv: "cbccc19b7c3691889ddb4071ae9706ad"
server_key_version: "encryptionkey@0"
I20260619 10:17:30.575654 6866 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260619 10:17:30.576364 6881 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 10:17:30.576639 6866 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260619 10:17:30.576738 6866 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781864224577058-825-0/minicluster-data/master-0/data,/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781864224577058-825-0/minicluster-data/master-0/wal
uuid: "bd146d3a022940a68274ddac0993c2e3"
format_stamp: "Formatted at 2026-06-19 10:17:30 on dist-test-slave-6pt2"
server_key: "5cab5e5c4ea69e1611a759f23ca4e18d"
server_key_iv: "cbccc19b7c3691889ddb4071ae9706ad"
server_key_version: "encryptionkey@0"
I20260619 10:17:30.576818 6866 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781864224577058-825-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781864224577058-825-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781864224577058-825-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 10:17:30.612679 6866 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 10:17:30.614482 6866 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 10:17:30.614674 6866 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 10:17:30.618386 6866 rpc_server.cc:307] RPC server started. Bound to: 127.0.206.126:38663
I20260619 10:17:30.618417 6933 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.206.126:38663 every 8 connection(s)
I20260619 10:17:30.618793 6866 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781864224577058-825-0/minicluster-data/master-0/data/info.pb
I20260619 10:17:30.619602 6934 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 10:17:30.621771 6934 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bd146d3a022940a68274ddac0993c2e3: Bootstrap starting.
I20260619 10:17:30.622464 6934 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bd146d3a022940a68274ddac0993c2e3: Neither blocks nor log segments found. Creating new log.
I20260619 10:17:30.622812 6934 log.cc:824] T 00000000000000000000000000000000 P bd146d3a022940a68274ddac0993c2e3: Log is configured to *not* fsync() on all Append() calls
I20260619 10:17:30.623615 6934 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bd146d3a022940a68274ddac0993c2e3: No bootstrap required, opened a new log
I20260619 10:17:30.625723 6934 raft_consensus.cc:348] T 00000000000000000000000000000000 P bd146d3a022940a68274ddac0993c2e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bd146d3a022940a68274ddac0993c2e3" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 38663 } }
I20260619 10:17:30.625873 6934 raft_consensus.cc:374] T 00000000000000000000000000000000 P bd146d3a022940a68274ddac0993c2e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 10:17:30.625917 6934 raft_consensus.cc:729] T 00000000000000000000000000000000 P bd146d3a022940a68274ddac0993c2e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd146d3a022940a68274ddac0993c2e3, State: Initialized, Role: FOLLOWER
I20260619 10:17:30.626055 6934 consensus_queue.cc:260] T 00000000000000000000000000000000 P bd146d3a022940a68274ddac0993c2e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bd146d3a022940a68274ddac0993c2e3" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 38663 } }
I20260619 10:17:30.626132 6934 raft_consensus.cc:388] T 00000000000000000000000000000000 P bd146d3a022940a68274ddac0993c2e3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 10:17:30.626171 6934 raft_consensus.cc:482] T 00000000000000000000000000000000 P bd146d3a022940a68274ddac0993c2e3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 10:17:30.626219 6934 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bd146d3a022940a68274ddac0993c2e3 [term 0 FOLLOWER]: Advancing to term 1
I20260619 10:17:30.627044 6934 raft_consensus.cc:504] T 00000000000000000000000000000000 P bd146d3a022940a68274ddac0993c2e3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bd146d3a022940a68274ddac0993c2e3" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 38663 } }
I20260619 10:17:30.627152 6934 leader_election.cc:302] T 00000000000000000000000000000000 P bd146d3a022940a68274ddac0993c2e3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bd146d3a022940a68274ddac0993c2e3; no voters:
I20260619 10:17:30.627339 6934 leader_election.cc:288] T 00000000000000000000000000000000 P bd146d3a022940a68274ddac0993c2e3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 10:17:30.627384 6937 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bd146d3a022940a68274ddac0993c2e3 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 10:17:30.627527 6937 raft_consensus.cc:686] T 00000000000000000000000000000000 P bd146d3a022940a68274ddac0993c2e3 [term 1 LEADER]: Becoming Leader. State: Replica: bd146d3a022940a68274ddac0993c2e3, State: Running, Role: LEADER
I20260619 10:17:30.627573 6934 sys_catalog.cc:564] T 00000000000000000000000000000000 P bd146d3a022940a68274ddac0993c2e3 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 10:17:30.627702 6937 consensus_queue.cc:237] T 00000000000000000000000000000000 P bd146d3a022940a68274ddac0993c2e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bd146d3a022940a68274ddac0993c2e3" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 38663 } }
I20260619 10:17:30.627996 825 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTU9OP6/build/release/bin/kudu as pid 6866
I20260619 10:17:30.628111 825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781864224577058-825-0/minicluster-data/master-0/wal/instance
I20260619 10:17:30.628108 6938 sys_catalog.cc:455] T 00000000000000000000000000000000 P bd146d3a022940a68274ddac0993c2e3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "bd146d3a022940a68274ddac0993c2e3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bd146d3a022940a68274ddac0993c2e3" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 38663 } } }
I20260619 10:17:30.628126 6939 sys_catalog.cc:455] T 00000000000000000000000000000000 P bd146d3a022940a68274ddac0993c2e3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader bd146d3a022940a68274ddac0993c2e3. Latest consensus state: current_term: 1 leader_uuid: "bd146d3a022940a68274ddac0993c2e3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bd146d3a022940a68274ddac0993c2e3" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 38663 } } }
I20260619 10:17:30.628208 6938 sys_catalog.cc:458] T 00000000000000000000000000000000 P bd146d3a022940a68274ddac0993c2e3 [sys.catalog]: This master's current role is: LEADER
I20260619 10:17:30.628216 6939 sys_catalog.cc:458] T 00000000000000000000000000000000 P bd146d3a022940a68274ddac0993c2e3 [sys.catalog]: This master's current role is: LEADER
I20260619 10:17:30.628304 825 external_mini_cluster.cc:1428] Setting key 76817476648cb43c3b8d73d8168ecba7
I20260619 10:17:30.628417 6946 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 10:17:30.628911 6946 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 10:17:30.630319 6946 catalog_manager.cc:1347] Generated new cluster ID: a825fc644b564a9897f7dc902ab7b888
I20260619 10:17:30.630368 6946 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Jun 19 10:17:30 dist-test-slave-6pt2 krb5kdc[6853](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864250, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.0.206.126@KRBTEST.COM
I20260619 10:17:30.640166 6956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:30.629007 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60176 (local address 127.0.206.126:38663)
0619 10:17:30.629594 (+ 587us) server_negotiation.cc:206] Beginning negotiation
0619 10:17:30.629600 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 10:17:30.629623 (+ 23us) server_negotiation.cc:407] Connection header received
0619 10:17:30.629703 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:17:30.629708 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:17:30.629747 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:17:30.629880 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:17:30.630592 (+ 712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:30.631975 (+ 1383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:30.633029 (+ 1054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:30.633630 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:30.633733 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:30.636420 (+ 2687us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 10:17:30.636424 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 10:17:30.636425 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 10:17:30.636436 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0619 10:17:30.638414 (+ 1978us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:17:30.638790 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:17:30.638793 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:17:30.638794 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:17:30.638921 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:17:30.639111 (+ 190us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:17:30.639112 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:17:30.639112 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:17:30.639450 (+ 338us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 10:17:30.639604 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0619 10:17:30.639679 (+ 75us) server_negotiation.cc:299] Negotiation successful
0619 10:17:30.639935 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":553,"thread_start_us":83,"threads_started":1}
I20260619 10:17:30.647949 6946 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 10:17:30.648418 6946 catalog_manager.cc:1504] Loading token signing keys...
I20260619 10:17:30.653401 6946 catalog_manager.cc:5868] T 00000000000000000000000000000000 P bd146d3a022940a68274ddac0993c2e3: Generated new TSK 0
I20260619 10:17:30.653573 6946 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.0.206.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.0.206.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.0.206.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781864224577058-825-0/krb5kdc/oryx_127.0.206.65.keytab.
Entry for principal oryx/127.0.206.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781864224577058-825-0/krb5kdc/oryx_127.0.206.65.keytab.
I20260619 10:17:30.679419 825 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTU9OP6/build/release/bin/kudu
/tmp/dist-test-taskTU9OP6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781864224577058-825-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781864224577058-825-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781864224577058-825-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781864224577058-825-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.206.65:0
--local_ip_for_outbound_sockets=127.0.206.65
--webserver_interface=127.0.206.65
--webserver_port=0
--tserver_master_addrs=127.0.206.126:38663
--keytab_file=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781864224577058-825-0/krb5kdc/oryx_127.0.206.65.keytab
--principal=oryx/127.0.206.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.206.84:33273
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781864224577058-825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781864224577058-825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781864224577058-825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 10:17:30.761373 6960 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 10:17:30.761561 6960 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 10:17:30.761583 6960 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 10:17:30.762723 6960 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 10:17:30.762763 6960 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 10:17:30.762789 6960 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.206.65
I20260619 10:17:30.764189 6960 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.206.84:33273
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781864224577058-825-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781864224577058-825-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781864224577058-825-0/krb5kdc/oryx_127.0.206.65.keytab
--principal=oryx/127.0.206.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.206.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781864224577058-825-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.206.65
--webserver_port=0
--tserver_master_addrs=127.0.206.126:38663
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6960
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.206.65
--log_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781864224577058-825-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 09:32:44 UTC on ccycloud.quasar-gbtldp.root.comops.site
build id 4455440
I20260619 10:17:30.764415 6960 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 10:17:30.764635 6960 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 10:17:30.770236 6966 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 10:17:30.770248 6965 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 10:17:30.770269 6968 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 10:17:30.770571 6960 server_base.cc:1034] running on GCE node
I20260619 10:17:30.770776 6960 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 10:17:30.771006 6960 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 10:17:30.772182 6960 hybrid_clock.cc:648] HybridClock initialized: now 1781864250772163 us; error 33 us; skew 500 ppm
Jun 19 10:17:30 dist-test-slave-6pt2 krb5kdc[6853](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864250, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.206.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 10:17:30.774492 6960 init.cc:377] Logged in from keytab as oryx/127.0.206.65@KRBTEST.COM (short username oryx)
I20260619 10:17:30.774932 6960 webserver.cc:460] Webserver started at http://127.0.206.65:39235/ using document root <none> and password file <none>
I20260619 10:17:30.775147 6960 fs_manager.cc:359] Metadata directory not provided
I20260619 10:17:30.775208 6960 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 10:17:30.775348 6960 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 10:17:30.776351 6960 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781864224577058-825-0/minicluster-data/ts-0/data/instance:
uuid: "67adfa875ae04601b7b8fceb7610166c"
format_stamp: "Formatted at 2026-06-19 10:17:30 on dist-test-slave-6pt2"
server_key: "e299581acb30e48f030e0da0d25b4d44"
server_key_iv: "8f80a3ec3d65fcefdf0d57ab42ec4c02"
server_key_version: "encryptionkey@0"
I20260619 10:17:30.776707 6960 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781864224577058-825-0/minicluster-data/ts-0/wal/instance:
uuid: "67adfa875ae04601b7b8fceb7610166c"
format_stamp: "Formatted at 2026-06-19 10:17:30 on dist-test-slave-6pt2"
server_key: "e299581acb30e48f030e0da0d25b4d44"
server_key_iv: "8f80a3ec3d65fcefdf0d57ab42ec4c02"
server_key_version: "encryptionkey@0"
I20260619 10:17:30.778077 6960 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260619 10:17:30.778793 6975 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 10:17:30.778951 6960 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 10:17:30.779042 6960 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781864224577058-825-0/minicluster-data/ts-0/data,/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781864224577058-825-0/minicluster-data/ts-0/wal
uuid: "67adfa875ae04601b7b8fceb7610166c"
format_stamp: "Formatted at 2026-06-19 10:17:30 on dist-test-slave-6pt2"
server_key: "e299581acb30e48f030e0da0d25b4d44"
server_key_iv: "8f80a3ec3d65fcefdf0d57ab42ec4c02"
server_key_version: "encryptionkey@0"
I20260619 10:17:30.779116 6960 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781864224577058-825-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781864224577058-825-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781864224577058-825-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 10:17:30.786339 6960 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 10:17:30.788059 6960 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 10:17:30.788204 6960 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 10:17:30.788400 6960 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 10:17:30.788681 6960 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 10:17:30.788729 6960 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 10:17:30.788769 6960 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 10:17:30.788796 6960 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 10:17:30.794341 6960 rpc_server.cc:307] RPC server started. Bound to: 127.0.206.65:33905
I20260619 10:17:30.794373 7088 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.206.65:33905 every 8 connection(s)
I20260619 10:17:30.794734 6960 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781864224577058-825-0/minicluster-data/ts-0/data/info.pb
Jun 19 10:17:30 dist-test-slave-6pt2 krb5kdc[6853](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864250, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.206.65@KRBTEST.COM for oryx/127.0.206.126@KRBTEST.COM
I20260619 10:17:30.803189 6956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:30.795373 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.65:50289 (local address 127.0.206.126:38663)
0619 10:17:30.795411 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0619 10:17:30.795412 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 10:17:30.795597 (+ 185us) server_negotiation.cc:407] Connection header received
0619 10:17:30.796111 (+ 514us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:17:30.796112 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:17:30.796125 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:17:30.796173 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:17:30.797059 (+ 886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:30.797564 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:30.798511 (+ 947us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:30.798798 (+ 287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:30.798879 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:30.800742 (+ 1863us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 10:17:30.800743 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 10:17:30.800744 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 10:17:30.800751 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 10:17:30.801863 (+ 1112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:17:30.802253 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:17:30.802254 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:17:30.802255 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:17:30.802313 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:17:30.802552 (+ 239us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:17:30.802553 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:17:30.802553 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:17:30.802699 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 10:17:30.802770 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0619 10:17:30.803021 (+ 251us) server_negotiation.cc:299] Negotiation successful
0619 10:17:30.803102 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260619 10:17:30.803356 7091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:30.795352 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:38663 (local address 127.0.206.65:50289)
0619 10:17:30.795483 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0619 10:17:30.795514 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0619 10:17:30.795995 (+ 481us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 10:17:30.796239 (+ 244us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 10:17:30.796242 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 10:17:30.796788 (+ 546us) client_negotiation.cc:189] Negotiated authn=SASL
0619 10:17:30.796964 (+ 176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:30.796969 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:30.797654 (+ 685us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:30.797655 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:30.798421 (+ 766us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:30.798423 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:30.798888 (+ 465us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:30.798889 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:30.799009 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:30.799406 (+ 397us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 10:17:30.799414 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 10:17:30.800638 (+ 1224us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 10:17:30.801961 (+ 1323us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 10:17:30.801968 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 10:17:30.801969 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 10:17:30.802173 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 10:17:30.802373 (+ 200us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 10:17:30.802374 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 10:17:30.802374 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 10:17:30.802479 (+ 105us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 10:17:30.802761 (+ 282us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 10:17:30.802767 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 10:17:30.802896 (+ 129us) client_negotiation.cc:769] Sending connection context
0619 10:17:30.802999 (+ 103us) client_negotiation.cc:240] Negotiation successful
0619 10:17:30.803143 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":99,"thread_start_us":76,"threads_started":1}
I20260619 10:17:30.803826 7089 heartbeater.cc:347] Connected to a master server at 127.0.206.126:38663
I20260619 10:17:30.803893 7089 heartbeater.cc:464] Registering TS with master...
I20260619 10:17:30.804093 7089 heartbeater.cc:511] Master 127.0.206.126:38663 requested a full tablet report, sending...
I20260619 10:17:30.804222 825 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTU9OP6/build/release/bin/kudu as pid 6960
I20260619 10:17:30.804316 825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781864224577058-825-0/minicluster-data/ts-0/wal/instance
I20260619 10:17:30.804539 825 external_mini_cluster.cc:1428] Setting key c8b37230e11acea52924278af871676e
I20260619 10:17:30.804651 6898 ts_manager.cc:194] Registered new tserver with Master: 67adfa875ae04601b7b8fceb7610166c (127.0.206.65:33905)
I20260619 10:17:30.805516 6898 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.0.206.65@KRBTEST.COM'} at 127.0.206.65:50289
WARNING: no policy specified for oryx/127.0.206.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.0.206.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.0.206.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781864224577058-825-0/krb5kdc/oryx_127.0.206.66.keytab.
Entry for principal oryx/127.0.206.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781864224577058-825-0/krb5kdc/oryx_127.0.206.66.keytab.
I20260619 10:17:30.826948 825 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTU9OP6/build/release/bin/kudu
/tmp/dist-test-taskTU9OP6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781864224577058-825-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781864224577058-825-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781864224577058-825-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781864224577058-825-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.206.66:0
--local_ip_for_outbound_sockets=127.0.206.66
--webserver_interface=127.0.206.66
--webserver_port=0
--tserver_master_addrs=127.0.206.126:38663
--keytab_file=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781864224577058-825-0/krb5kdc/oryx_127.0.206.66.keytab
--principal=oryx/127.0.206.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.206.84:33273
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781864224577058-825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781864224577058-825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781864224577058-825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 10:17:30.906507 7094 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 10:17:30.906713 7094 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 10:17:30.906735 7094 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 10:17:30.907833 7094 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 10:17:30.907872 7094 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 10:17:30.907925 7094 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.206.66
I20260619 10:17:30.909303 7094 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.206.84:33273
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781864224577058-825-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781864224577058-825-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781864224577058-825-0/krb5kdc/oryx_127.0.206.66.keytab
--principal=oryx/127.0.206.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.206.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781864224577058-825-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.206.66
--webserver_port=0
--tserver_master_addrs=127.0.206.126:38663
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7094
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.206.66
--log_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781864224577058-825-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 09:32:44 UTC on ccycloud.quasar-gbtldp.root.comops.site
build id 4455440
I20260619 10:17:30.909493 7094 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 10:17:30.909682 7094 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 10:17:30.919653 7099 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 10:17:30.919673 7102 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 10:17:30.919818 7100 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 10:17:30.920861 7094 server_base.cc:1034] running on GCE node
I20260619 10:17:30.921092 7094 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 10:17:30.921362 7094 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 10:17:30.922513 7094 hybrid_clock.cc:648] HybridClock initialized: now 1781864250922500 us; error 28 us; skew 500 ppm
Jun 19 10:17:30 dist-test-slave-6pt2 krb5kdc[6853](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864250, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.206.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 10:17:30.924913 7094 init.cc:377] Logged in from keytab as oryx/127.0.206.66@KRBTEST.COM (short username oryx)
I20260619 10:17:30.925295 7094 webserver.cc:460] Webserver started at http://127.0.206.66:38305/ using document root <none> and password file <none>
I20260619 10:17:30.925495 7094 fs_manager.cc:359] Metadata directory not provided
I20260619 10:17:30.925536 7094 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 10:17:30.925666 7094 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 10:17:30.926714 7094 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781864224577058-825-0/minicluster-data/ts-1/data/instance:
uuid: "04a062435b8049fa92d4f33097d4e8a2"
format_stamp: "Formatted at 2026-06-19 10:17:30 on dist-test-slave-6pt2"
server_key: "bdf4ff498a51788d1fe38f0489c5e00a"
server_key_iv: "70df18662e03f5d78aa0770ca5d89f9e"
server_key_version: "encryptionkey@0"
I20260619 10:17:30.927065 7094 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781864224577058-825-0/minicluster-data/ts-1/wal/instance:
uuid: "04a062435b8049fa92d4f33097d4e8a2"
format_stamp: "Formatted at 2026-06-19 10:17:30 on dist-test-slave-6pt2"
server_key: "bdf4ff498a51788d1fe38f0489c5e00a"
server_key_iv: "70df18662e03f5d78aa0770ca5d89f9e"
server_key_version: "encryptionkey@0"
I20260619 10:17:30.928315 7094 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 10:17:30.928972 7109 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 10:17:30.929142 7094 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 10:17:30.929224 7094 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781864224577058-825-0/minicluster-data/ts-1/data,/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781864224577058-825-0/minicluster-data/ts-1/wal
uuid: "04a062435b8049fa92d4f33097d4e8a2"
format_stamp: "Formatted at 2026-06-19 10:17:30 on dist-test-slave-6pt2"
server_key: "bdf4ff498a51788d1fe38f0489c5e00a"
server_key_iv: "70df18662e03f5d78aa0770ca5d89f9e"
server_key_version: "encryptionkey@0"
I20260619 10:17:30.929275 7094 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781864224577058-825-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781864224577058-825-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781864224577058-825-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 10:17:30.942562 7094 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 10:17:30.944679 7094 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 10:17:30.944782 7094 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 10:17:30.944953 7094 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 10:17:30.945176 7094 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 10:17:30.945205 7094 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 10:17:30.945225 7094 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 10:17:30.945238 7094 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 10:17:30.950742 7094 rpc_server.cc:307] RPC server started. Bound to: 127.0.206.66:37571
I20260619 10:17:30.950788 7222 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.206.66:37571 every 8 connection(s)
I20260619 10:17:30.951089 7094 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781864224577058-825-0/minicluster-data/ts-1/data/info.pb
I20260619 10:17:30.951367 825 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTU9OP6/build/release/bin/kudu as pid 7094
I20260619 10:17:30.951478 825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781864224577058-825-0/minicluster-data/ts-1/wal/instance
I20260619 10:17:30.951676 825 external_mini_cluster.cc:1428] Setting key 97ded563a07b52a735c9a52ea3efca20
Jun 19 10:17:30 dist-test-slave-6pt2 krb5kdc[6853](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864250, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.206.66@KRBTEST.COM for oryx/127.0.206.126@KRBTEST.COM
WARNING: no policy specified for oryx/127.0.206.67@KRBTEST.COM; defaulting to no policy
I20260619 10:17:30.960875 6956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:30.951713 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.66:60861 (local address 127.0.206.126:38663)
0619 10:17:30.951879 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0619 10:17:30.951881 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 10:17:30.951964 (+ 83us) server_negotiation.cc:407] Connection header received
0619 10:17:30.952629 (+ 665us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:17:30.952629 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:17:30.952638 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:17:30.952673 (+ 35us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:17:30.953927 (+ 1254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:30.954728 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:30.955679 (+ 951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:30.956011 (+ 332us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:30.956085 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:30.958040 (+ 1955us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 10:17:30.958042 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 10:17:30.958043 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 10:17:30.958051 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 10:17:30.959644 (+ 1593us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:17:30.959997 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:17:30.959998 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:17:30.959998 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:17:30.960058 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:17:30.960319 (+ 261us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:17:30.960320 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:17:30.960320 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:17:30.960475 (+ 155us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 10:17:30.960542 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0619 10:17:30.960753 (+ 211us) server_negotiation.cc:299] Negotiation successful
0619 10:17:30.960814 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":140}
I20260619 10:17:30.961037 7225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:30.951691 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:38663 (local address 127.0.206.66:60861)
0619 10:17:30.951837 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0619 10:17:30.951882 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0619 10:17:30.952534 (+ 652us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 10:17:30.952750 (+ 216us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 10:17:30.952754 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 10:17:30.953558 (+ 804us) client_negotiation.cc:189] Negotiated authn=SASL
0619 10:17:30.953824 (+ 266us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:30.953831 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:30.954816 (+ 985us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:30.954817 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:30.955589 (+ 772us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:30.955591 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:30.956100 (+ 509us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:30.956100 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:30.956196 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:30.956658 (+ 462us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 10:17:30.956666 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 10:17:30.957946 (+ 1280us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 10:17:30.959725 (+ 1779us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 10:17:30.959730 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 10:17:30.959732 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 10:17:30.959928 (+ 196us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 10:17:30.960122 (+ 194us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 10:17:30.960123 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 10:17:30.960123 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 10:17:30.960257 (+ 134us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 10:17:30.960541 (+ 284us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 10:17:30.960546 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 10:17:30.960648 (+ 102us) client_negotiation.cc:769] Sending connection context
0619 10:17:30.960727 (+ 79us) client_negotiation.cc:240] Negotiation successful
0619 10:17:30.960832 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":71,"threads_started":1}
I20260619 10:17:30.961592 7223 heartbeater.cc:347] Connected to a master server at 127.0.206.126:38663
I20260619 10:17:30.961678 7223 heartbeater.cc:464] Registering TS with master...
I20260619 10:17:30.961863 7223 heartbeater.cc:511] Master 127.0.206.126:38663 requested a full tablet report, sending...
I20260619 10:17:30.962246 6898 ts_manager.cc:194] Registered new tserver with Master: 04a062435b8049fa92d4f33097d4e8a2 (127.0.206.66:37571)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.0.206.67@KRBTEST.COM" created.
I20260619 10:17:30.962733 6898 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.0.206.66@KRBTEST.COM'} at 127.0.206.66:60861
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.0.206.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781864224577058-825-0/krb5kdc/oryx_127.0.206.67.keytab.
Entry for principal oryx/127.0.206.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781864224577058-825-0/krb5kdc/oryx_127.0.206.67.keytab.
I20260619 10:17:30.974852 825 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTU9OP6/build/release/bin/kudu
/tmp/dist-test-taskTU9OP6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781864224577058-825-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781864224577058-825-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781864224577058-825-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781864224577058-825-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.206.67:0
--local_ip_for_outbound_sockets=127.0.206.67
--webserver_interface=127.0.206.67
--webserver_port=0
--tserver_master_addrs=127.0.206.126:38663
--keytab_file=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781864224577058-825-0/krb5kdc/oryx_127.0.206.67.keytab
--principal=oryx/127.0.206.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.206.84:33273
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781864224577058-825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781864224577058-825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781864224577058-825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 10:17:31.052263 7228 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 10:17:31.052440 7228 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 10:17:31.052461 7228 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 10:17:31.053563 7228 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 10:17:31.053602 7228 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 10:17:31.053649 7228 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.206.67
I20260619 10:17:31.055014 7228 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.206.84:33273
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781864224577058-825-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781864224577058-825-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781864224577058-825-0/krb5kdc/oryx_127.0.206.67.keytab
--principal=oryx/127.0.206.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.206.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781864224577058-825-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.206.67
--webserver_port=0
--tserver_master_addrs=127.0.206.126:38663
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7228
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.206.67
--log_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781864224577058-825-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 09:32:44 UTC on ccycloud.quasar-gbtldp.root.comops.site
build id 4455440
I20260619 10:17:31.055198 7228 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 10:17:31.055405 7228 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 10:17:31.060031 7234 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 10:17:31.060148 7228 server_base.cc:1034] running on GCE node
W20260619 10:17:31.060041 7236 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 10:17:31.060042 7233 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 10:17:31.060420 7228 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 10:17:31.060658 7228 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 10:17:31.061801 7228 hybrid_clock.cc:648] HybridClock initialized: now 1781864251061791 us; error 27 us; skew 500 ppm
Jun 19 10:17:31 dist-test-slave-6pt2 krb5kdc[6853](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864251, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.206.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 10:17:31.064150 7228 init.cc:377] Logged in from keytab as oryx/127.0.206.67@KRBTEST.COM (short username oryx)
I20260619 10:17:31.064538 7228 webserver.cc:460] Webserver started at http://127.0.206.67:37187/ using document root <none> and password file <none>
I20260619 10:17:31.064759 7228 fs_manager.cc:359] Metadata directory not provided
I20260619 10:17:31.064821 7228 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 10:17:31.064960 7228 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 10:17:31.065891 7228 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781864224577058-825-0/minicluster-data/ts-2/data/instance:
uuid: "9c6d725a83a94c8da04ad1daed4d3816"
format_stamp: "Formatted at 2026-06-19 10:17:31 on dist-test-slave-6pt2"
server_key: "da5fcbd3c4224fa4bc075103d9d08618"
server_key_iv: "707865a767bc195eac029631578c9012"
server_key_version: "encryptionkey@0"
I20260619 10:17:31.066210 7228 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781864224577058-825-0/minicluster-data/ts-2/wal/instance:
uuid: "9c6d725a83a94c8da04ad1daed4d3816"
format_stamp: "Formatted at 2026-06-19 10:17:31 on dist-test-slave-6pt2"
server_key: "da5fcbd3c4224fa4bc075103d9d08618"
server_key_iv: "707865a767bc195eac029631578c9012"
server_key_version: "encryptionkey@0"
I20260619 10:17:31.067528 7228 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260619 10:17:31.068467 7243 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 10:17:31.068653 7228 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260619 10:17:31.068774 7228 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781864224577058-825-0/minicluster-data/ts-2/data,/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781864224577058-825-0/minicluster-data/ts-2/wal
uuid: "9c6d725a83a94c8da04ad1daed4d3816"
format_stamp: "Formatted at 2026-06-19 10:17:31 on dist-test-slave-6pt2"
server_key: "da5fcbd3c4224fa4bc075103d9d08618"
server_key_iv: "707865a767bc195eac029631578c9012"
server_key_version: "encryptionkey@0"
I20260619 10:17:31.068838 7228 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781864224577058-825-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781864224577058-825-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781864224577058-825-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 10:17:31.081077 7228 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 10:17:31.083274 7228 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 10:17:31.083398 7228 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 10:17:31.083571 7228 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 10:17:31.083787 7228 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 10:17:31.083817 7228 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 10:17:31.083837 7228 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 10:17:31.083851 7228 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 10:17:31.089280 7228 rpc_server.cc:307] RPC server started. Bound to: 127.0.206.67:43923
I20260619 10:17:31.089362 7356 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.206.67:43923 every 8 connection(s)
I20260619 10:17:31.089663 7228 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781864224577058-825-0/minicluster-data/ts-2/data/info.pb
Jun 19 10:17:31 dist-test-slave-6pt2 krb5kdc[6853](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864251, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.206.67@KRBTEST.COM for oryx/127.0.206.126@KRBTEST.COM
I20260619 10:17:31.098297 6956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:31.090315 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.67:45377 (local address 127.0.206.126:38663)
0619 10:17:31.090365 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0619 10:17:31.090367 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 10:17:31.090619 (+ 252us) server_negotiation.cc:407] Connection header received
0619 10:17:31.091115 (+ 496us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:17:31.091117 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:17:31.091126 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:17:31.091184 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:17:31.092043 (+ 859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:31.092539 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:31.093483 (+ 944us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:31.093664 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:31.093720 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:31.095638 (+ 1918us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 10:17:31.095640 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 10:17:31.095640 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 10:17:31.095648 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 10:17:31.096821 (+ 1173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:17:31.097172 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:17:31.097173 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:17:31.097173 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:17:31.097221 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:17:31.097562 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:17:31.097563 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:17:31.097563 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:17:31.097718 (+ 155us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 10:17:31.097788 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0619 10:17:31.098089 (+ 301us) server_negotiation.cc:299] Negotiation successful
0619 10:17:31.098189 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260619 10:17:31.098438 7359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:31.090300 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:38663 (local address 127.0.206.67:45377)
0619 10:17:31.090465 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0619 10:17:31.090496 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0619 10:17:31.091000 (+ 504us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 10:17:31.091254 (+ 254us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 10:17:31.091257 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 10:17:31.091791 (+ 534us) client_negotiation.cc:189] Negotiated authn=SASL
0619 10:17:31.091966 (+ 175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:31.091970 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:31.092613 (+ 643us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:31.092613 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:31.093401 (+ 788us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:31.093403 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:31.093733 (+ 330us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:31.093734 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:31.093823 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:31.094220 (+ 397us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 10:17:31.094228 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 10:17:31.095559 (+ 1331us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 10:17:31.096902 (+ 1343us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 10:17:31.096907 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 10:17:31.096908 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 10:17:31.097113 (+ 205us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 10:17:31.097327 (+ 214us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 10:17:31.097328 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 10:17:31.097328 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 10:17:31.097465 (+ 137us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 10:17:31.097786 (+ 321us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 10:17:31.097791 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 10:17:31.097938 (+ 147us) client_negotiation.cc:769] Sending connection context
0619 10:17:31.098048 (+ 110us) client_negotiation.cc:240] Negotiation successful
0619 10:17:31.098213 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":91,"threads_started":1}
I20260619 10:17:31.098867 7357 heartbeater.cc:347] Connected to a master server at 127.0.206.126:38663
I20260619 10:17:31.098961 7357 heartbeater.cc:464] Registering TS with master...
I20260619 10:17:31.099126 7357 heartbeater.cc:511] Master 127.0.206.126:38663 requested a full tablet report, sending...
I20260619 10:17:31.099262 825 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTU9OP6/build/release/bin/kudu as pid 7228
I20260619 10:17:31.099362 825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781864224577058-825-0/minicluster-data/ts-2/wal/instance
I20260619 10:17:31.099541 6898 ts_manager.cc:194] Registered new tserver with Master: 9c6d725a83a94c8da04ad1daed4d3816 (127.0.206.67:43923)
I20260619 10:17:31.099563 825 external_mini_cluster.cc:1428] Setting key f075e1f9ee08658e962d7b29f3faac32
I20260619 10:17:31.099907 6898 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.0.206.67@KRBTEST.COM'} at 127.0.206.67:45377
I20260619 10:17:31.100404 825 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.0.206.125@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.0.206.125@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.0.206.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781864224577058-825-0/krb5kdc/oryx_127.0.206.125.keytab.
Entry for principal oryx/127.0.206.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781864224577058-825-0/krb5kdc/oryx_127.0.206.125.keytab.
I20260619 10:17:31.123400 825 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTU9OP6/build/release/bin/kudu
/tmp/dist-test-taskTU9OP6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781864224577058-825-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781864224577058-825-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781864224577058-825-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781864224577058-825-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.206.125:40043
--webserver_interface=127.0.206.125
--webserver_port=0
--keytab_file=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781864224577058-825-0/krb5kdc/oryx_127.0.206.125.keytab
--principal=oryx/127.0.206.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.206.84:33273
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.206.126:38663,127.0.206.125:40043
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781864224577058-825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781864224577058-825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781864224577058-825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 10:17:31.202791 7362 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 10:17:31.203024 7362 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 10:17:31.203059 7362 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 10:17:31.204219 7362 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 10:17:31.204283 7362 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 10:17:31.204306 7362 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 10:17:31.204327 7362 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 10:17:31.204355 7362 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 10:17:31.205789 7362 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.206.84:33273
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781864224577058-825-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781864224577058-825-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.0.206.126:38663,127.0.206.125:40043
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781864224577058-825-0/krb5kdc/oryx_127.0.206.125.keytab
--principal=oryx/127.0.206.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.206.125:40043
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781864224577058-825-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.0.206.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7362
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781864224577058-825-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 09:32:44 UTC on ccycloud.quasar-gbtldp.root.comops.site
build id 4455440
W20260619 10:17:31.205937 7362 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.0.206.126:38663,127.0.206.125:40043'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260619 10:17:31.206089 7362 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 10:17:31.206291 7362 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 10:17:31.210913 7370 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 10:17:31.210952 7368 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 10:17:31.211014 7362 server_base.cc:1034] running on GCE node
W20260619 10:17:31.210883 7367 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 10:17:31.211298 7362 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 10:17:31.211601 7362 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 10:17:31.212757 7362 hybrid_clock.cc:648] HybridClock initialized: now 1781864251212744 us; error 26 us; skew 500 ppm
Jun 19 10:17:31 dist-test-slave-6pt2 krb5kdc[6853](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864251, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.206.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 10:17:31.215139 7362 init.cc:377] Logged in from keytab as oryx/127.0.206.125@KRBTEST.COM (short username oryx)
I20260619 10:17:31.215538 7362 webserver.cc:460] Webserver started at http://127.0.206.125:39145/ using document root <none> and password file <none>
I20260619 10:17:31.215752 7362 fs_manager.cc:359] Metadata directory not provided
I20260619 10:17:31.215814 7362 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 10:17:31.215945 7362 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 10:17:31.216924 7362 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781864224577058-825-0/minicluster-data/master-1/data/instance:
uuid: "cd27214b90874d35b38271d228518b20"
format_stamp: "Formatted at 2026-06-19 10:17:31 on dist-test-slave-6pt2"
server_key: "83b12de1595d168f62089f11371cdded"
server_key_iv: "0713785e593d35c7409d1dda47e86324"
server_key_version: "encryptionkey@0"
I20260619 10:17:31.217317 7362 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781864224577058-825-0/minicluster-data/master-1/wal/instance:
uuid: "cd27214b90874d35b38271d228518b20"
format_stamp: "Formatted at 2026-06-19 10:17:31 on dist-test-slave-6pt2"
server_key: "83b12de1595d168f62089f11371cdded"
server_key_iv: "0713785e593d35c7409d1dda47e86324"
server_key_version: "encryptionkey@0"
I20260619 10:17:31.218529 7362 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260619 10:17:31.219224 7377 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 10:17:31.219386 7362 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260619 10:17:31.219477 7362 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781864224577058-825-0/minicluster-data/master-1/data,/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781864224577058-825-0/minicluster-data/master-1/wal
uuid: "cd27214b90874d35b38271d228518b20"
format_stamp: "Formatted at 2026-06-19 10:17:31 on dist-test-slave-6pt2"
server_key: "83b12de1595d168f62089f11371cdded"
server_key_iv: "0713785e593d35c7409d1dda47e86324"
server_key_version: "encryptionkey@0"
I20260619 10:17:31.219556 7362 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781864224577058-825-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781864224577058-825-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781864224577058-825-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 10:17:31.237815 7362 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 10:17:31.239539 7362 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 10:17:31.239693 7362 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 10:17:31.243353 7362 rpc_server.cc:307] RPC server started. Bound to: 127.0.206.125:40043
I20260619 10:17:31.243382 7429 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.206.125:40043 every 8 connection(s)
I20260619 10:17:31.243701 7362 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781864224577058-825-0/minicluster-data/master-1/data/info.pb
I20260619 10:17:31.244562 7430 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 10:17:31.245950 7430 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.206.126" port: 38663 } has no permanent_uuid. Determining permanent_uuid...
I20260619 10:17:31.247480 825 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTU9OP6/build/release/bin/kudu as pid 7362
Jun 19 10:17:31 dist-test-slave-6pt2 krb5kdc[6853](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864251, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.206.125@KRBTEST.COM for oryx/127.0.206.126@KRBTEST.COM
Jun 19 10:17:31 dist-test-slave-6pt2 krb5kdc[6853](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864250, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.0.206.125@KRBTEST.COM
I20260619 10:17:31.255383 6956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:31.246580 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60178 (local address 127.0.206.126:38663)
0619 10:17:31.246672 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0619 10:17:31.246674 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 10:17:31.246826 (+ 152us) server_negotiation.cc:407] Connection header received
0619 10:17:31.247240 (+ 414us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:17:31.247241 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:17:31.247251 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:17:31.247302 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:17:31.247955 (+ 653us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:31.248767 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:31.250108 (+ 1341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:31.250386 (+ 278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:31.250453 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:31.252650 (+ 2197us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 10:17:31.252652 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 10:17:31.252652 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 10:17:31.252659 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 10:17:31.254014 (+ 1355us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:17:31.254387 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:17:31.254388 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:17:31.254389 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:17:31.254443 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:17:31.254707 (+ 264us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:17:31.254708 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:17:31.254708 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:17:31.254878 (+ 170us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 10:17:31.254953 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0619 10:17:31.255210 (+ 257us) server_negotiation.cc:299] Negotiation successful
0619 10:17:31.255286 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260619 10:17:31.255617 7432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:31.246543 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:38663 (local address 127.0.0.1:60178)
0619 10:17:31.246723 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0619 10:17:31.246755 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0619 10:17:31.247162 (+ 407us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 10:17:31.247359 (+ 197us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 10:17:31.247362 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 10:17:31.247634 (+ 272us) client_negotiation.cc:189] Negotiated authn=SASL
0619 10:17:31.247855 (+ 221us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:31.247861 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:31.248892 (+ 1031us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:31.248893 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:31.250010 (+ 1117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:31.250013 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:31.250460 (+ 447us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:31.250461 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:31.250553 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:31.251104 (+ 551us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 10:17:31.251113 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0619 10:17:31.252567 (+ 1454us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 10:17:31.254093 (+ 1526us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 10:17:31.254098 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 10:17:31.254100 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 10:17:31.254307 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 10:17:31.254508 (+ 201us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 10:17:31.254509 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 10:17:31.254510 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 10:17:31.254651 (+ 141us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 10:17:31.254944 (+ 293us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 10:17:31.254949 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 10:17:31.255076 (+ 127us) client_negotiation.cc:769] Sending connection context
0619 10:17:31.255179 (+ 103us) client_negotiation.cc:240] Negotiation successful
0619 10:17:31.255357 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":102,"threads_started":1}
I20260619 10:17:31.256249 7430 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.206.125" port: 40043 } has no permanent_uuid. Determining permanent_uuid...
I20260619 10:17:31.256996 7434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:31.247941 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38780 (local address 127.0.206.125:40043)
0619 10:17:31.248094 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0619 10:17:31.248102 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 10:17:31.248116 (+ 14us) server_negotiation.cc:407] Connection header received
0619 10:17:31.248133 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:17:31.248134 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:17:31.248148 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:17:31.248209 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:17:31.248874 (+ 665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:31.250027 (+ 1153us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:31.250970 (+ 943us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:31.251387 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:31.251460 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:31.253856 (+ 2396us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 10:17:31.253863 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 10:17:31.253863 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 10:17:31.253870 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 10:17:31.255395 (+ 1525us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:17:31.255850 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:17:31.255852 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:17:31.255852 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:17:31.255931 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:17:31.256146 (+ 215us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:17:31.256147 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:17:31.256147 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:17:31.256294 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 10:17:31.256437 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0619 10:17:31.256853 (+ 416us) server_negotiation.cc:299] Negotiation successful
0619 10:17:31.256932 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":127,"thread_start_us":90,"threads_started":1}
Jun 19 10:17:31 dist-test-slave-6pt2 krb5kdc[6853](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864251, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.206.125@KRBTEST.COM for oryx/127.0.206.125@KRBTEST.COM
I20260619 10:17:31.262392 7432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:31.256492 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.125:40043 (local address 127.0.0.1:38794)
0619 10:17:31.256553 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0619 10:17:31.256563 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 10:17:31.256597 (+ 34us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 10:17:31.256841 (+ 244us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 10:17:31.256842 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 10:17:31.256964 (+ 122us) client_negotiation.cc:189] Negotiated authn=SASL
0619 10:17:31.257001 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:31.257002 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:31.257658 (+ 656us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:31.257659 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:31.258114 (+ 455us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:31.258115 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:31.258488 (+ 373us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:31.258489 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:31.258555 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:31.258862 (+ 307us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 10:17:31.258866 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 10:17:31.259881 (+ 1015us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 10:17:31.261638 (+ 1757us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 10:17:31.261639 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 10:17:31.261639 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 10:17:31.261795 (+ 156us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 10:17:31.261964 (+ 169us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 10:17:31.261964 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 10:17:31.261964 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 10:17:31.262016 (+ 52us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 10:17:31.262197 (+ 181us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 10:17:31.262197 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 10:17:31.262228 (+ 31us) client_negotiation.cc:769] Sending connection context
0619 10:17:31.262276 (+ 48us) client_negotiation.cc:240] Negotiation successful
0619 10:17:31.262340 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":41}
I20260619 10:17:31.262501 7435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:31.256556 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38794 (local address 127.0.206.125:40043)
0619 10:17:31.256733 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0619 10:17:31.256735 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 10:17:31.256740 (+ 5us) server_negotiation.cc:407] Connection header received
0619 10:17:31.256753 (+ 13us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:17:31.256753 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:17:31.256760 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:17:31.256826 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:17:31.257064 (+ 238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:31.257548 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:31.258210 (+ 662us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:31.258416 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:31.258488 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:31.259950 (+ 1462us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 10:17:31.259951 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 10:17:31.259951 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 10:17:31.259956 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0619 10:17:31.261564 (+ 1608us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:17:31.261847 (+ 283us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:17:31.261848 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:17:31.261867 (+ 19us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:17:31.261912 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:17:31.262052 (+ 140us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:17:31.262053 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:17:31.262053 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:17:31.262147 (+ 94us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 10:17:31.262191 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0619 10:17:31.262289 (+ 98us) server_negotiation.cc:299] Negotiation successful
0619 10:17:31.262382 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":39,"server-negotiator.queue_time_us":162,"thread_start_us":74,"threads_started":1}
I20260619 10:17:31.263563 7430 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cd27214b90874d35b38271d228518b20: Bootstrap starting.
I20260619 10:17:31.264129 7430 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cd27214b90874d35b38271d228518b20: Neither blocks nor log segments found. Creating new log.
I20260619 10:17:31.264387 7430 log.cc:824] T 00000000000000000000000000000000 P cd27214b90874d35b38271d228518b20: Log is configured to *not* fsync() on all Append() calls
I20260619 10:17:31.265000 7430 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cd27214b90874d35b38271d228518b20: No bootstrap required, opened a new log
I20260619 10:17:31.266258 7430 raft_consensus.cc:348] T 00000000000000000000000000000000 P cd27214b90874d35b38271d228518b20 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd146d3a022940a68274ddac0993c2e3" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 38663 } } peers { permanent_uuid: "cd27214b90874d35b38271d228518b20" member_type: VOTER last_known_addr { host: "127.0.206.125" port: 40043 } }
I20260619 10:17:31.266383 7430 raft_consensus.cc:374] T 00000000000000000000000000000000 P cd27214b90874d35b38271d228518b20 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 10:17:31.266427 7430 raft_consensus.cc:729] T 00000000000000000000000000000000 P cd27214b90874d35b38271d228518b20 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cd27214b90874d35b38271d228518b20, State: Initialized, Role: FOLLOWER
I20260619 10:17:31.266561 7430 consensus_queue.cc:260] T 00000000000000000000000000000000 P cd27214b90874d35b38271d228518b20 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd146d3a022940a68274ddac0993c2e3" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 38663 } } peers { permanent_uuid: "cd27214b90874d35b38271d228518b20" member_type: VOTER last_known_addr { host: "127.0.206.125" port: 40043 } }
I20260619 10:17:31.266834 7430 sys_catalog.cc:564] T 00000000000000000000000000000000 P cd27214b90874d35b38271d228518b20 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 10:17:31.266832 7437 sys_catalog.cc:455] T 00000000000000000000000000000000 P cd27214b90874d35b38271d228518b20 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd146d3a022940a68274ddac0993c2e3" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 38663 } } peers { permanent_uuid: "cd27214b90874d35b38271d228518b20" member_type: VOTER last_known_addr { host: "127.0.206.125" port: 40043 } } }
I20260619 10:17:31.266964 7437 sys_catalog.cc:458] T 00000000000000000000000000000000 P cd27214b90874d35b38271d228518b20 [sys.catalog]: This master's current role is: FOLLOWER
W20260619 10:17:31.268428 7448 catalog_manager.cc:1558] T 00000000000000000000000000000000 P cd27214b90874d35b38271d228518b20: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 10:17:31.268481 7448 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 10:17:31.269770 7362 master_runner.cc:428] Detected that this master cd27214b90874d35b38271d228518b20 is joining an existing cluster
I20260619 10:17:31.269819 7362 master_runner.cc:432] Initiating AddMaster RPC to add 127.0.206.125:40043
I20260619 10:17:31.276700 7435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:31.271002 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38798 (local address 127.0.206.125:40043)
0619 10:17:31.271036 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0619 10:17:31.271037 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 10:17:31.271126 (+ 89us) server_negotiation.cc:407] Connection header received
0619 10:17:31.271176 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:17:31.271177 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:17:31.271185 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:17:31.271226 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:17:31.271577 (+ 351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:31.272034 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:31.272840 (+ 806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:31.273048 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:31.273122 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:31.273951 (+ 829us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 10:17:31.273952 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 10:17:31.273953 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 10:17:31.273959 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 10:17:31.275595 (+ 1636us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:17:31.275938 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:17:31.275939 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:17:31.275940 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:17:31.275986 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:17:31.276187 (+ 201us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:17:31.276188 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:17:31.276188 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:17:31.276283 (+ 95us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 10:17:31.276348 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0619 10:17:31.276479 (+ 131us) server_negotiation.cc:299] Negotiation successful
0619 10:17:31.276601 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":5,"server-negotiator.queue_time_us":17}
I20260619 10:17:31.276932 7455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:31.270945 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.125:40043 (local address 127.0.0.1:38798)
0619 10:17:31.271081 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0619 10:17:31.271089 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0619 10:17:31.271135 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 10:17:31.271255 (+ 120us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 10:17:31.271255 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 10:17:31.271419 (+ 164us) client_negotiation.cc:189] Negotiated authn=SASL
0619 10:17:31.271472 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:31.271473 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:31.272104 (+ 631us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:31.272104 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:31.272757 (+ 653us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:31.272758 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:31.273139 (+ 381us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:31.273140 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:31.273208 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:31.273597 (+ 389us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 10:17:31.273601 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 10:17:31.273859 (+ 258us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 10:17:31.275673 (+ 1814us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 10:17:31.275674 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 10:17:31.275674 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 10:17:31.275854 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 10:17:31.276078 (+ 224us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 10:17:31.276079 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 10:17:31.276079 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 10:17:31.276129 (+ 50us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 10:17:31.276340 (+ 211us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 10:17:31.276341 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 10:17:31.276373 (+ 32us) client_negotiation.cc:769] Sending connection context
0619 10:17:31.276461 (+ 88us) client_negotiation.cc:240] Negotiation successful
0619 10:17:31.276700 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"mutex_wait_us":104,"thread_start_us":70,"threads_started":1}
I20260619 10:17:31.277570 6956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:31.271117 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60194 (local address 127.0.206.126:38663)
0619 10:17:31.271153 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0619 10:17:31.271156 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 10:17:31.271163 (+ 7us) server_negotiation.cc:407] Connection header received
0619 10:17:31.271182 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:17:31.271183 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:17:31.271192 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:17:31.271250 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:17:31.271691 (+ 441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:31.272418 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:31.273452 (+ 1034us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:31.273732 (+ 280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:31.273772 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:31.274804 (+ 1032us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 10:17:31.274805 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 10:17:31.274805 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 10:17:31.274812 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 10:17:31.276358 (+ 1546us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:17:31.276816 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:17:31.276817 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:17:31.276818 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:17:31.276874 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:17:31.277056 (+ 182us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:17:31.277057 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:17:31.277058 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:17:31.277181 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 10:17:31.277239 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0619 10:17:31.277449 (+ 210us) server_negotiation.cc:299] Negotiation successful
0619 10:17:31.277514 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260619 10:17:31.277631 7454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:31.270763 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:38663 (local address 127.0.0.1:60194)
0619 10:17:31.270893 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0619 10:17:31.270903 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 10:17:31.270945 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 10:17:31.271290 (+ 345us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 10:17:31.271291 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 10:17:31.271527 (+ 236us) client_negotiation.cc:189] Negotiated authn=SASL
0619 10:17:31.271580 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:31.271582 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:31.272512 (+ 930us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:31.272513 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:31.273337 (+ 824us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:31.273340 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:31.273844 (+ 504us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:31.273845 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:31.273908 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:31.274283 (+ 375us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 10:17:31.274288 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 10:17:31.274730 (+ 442us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 10:17:31.276448 (+ 1718us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 10:17:31.276449 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 10:17:31.276449 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 10:17:31.276737 (+ 288us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 10:17:31.276941 (+ 204us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 10:17:31.276942 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 10:17:31.276942 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 10:17:31.276995 (+ 53us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 10:17:31.277265 (+ 270us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 10:17:31.277265 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 10:17:31.277311 (+ 46us) client_negotiation.cc:769] Sending connection context
0619 10:17:31.277420 (+ 109us) client_negotiation.cc:240] Negotiation successful
0619 10:17:31.277510 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":71,"threads_started":1}
Jun 19 10:17:31 dist-test-slave-6pt2 krb5kdc[6853](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864250, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.206.126@KRBTEST.COM for oryx/127.0.206.125@KRBTEST.COM
I20260619 10:17:31.285871 7435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:31.279621 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38808 (local address 127.0.206.125:40043)
0619 10:17:31.279655 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0619 10:17:31.279656 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 10:17:31.279790 (+ 134us) server_negotiation.cc:407] Connection header received
0619 10:17:31.279888 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:17:31.279889 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:17:31.279898 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:17:31.279951 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:17:31.280333 (+ 382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:31.280777 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:31.281507 (+ 730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:31.281696 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:31.281768 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:31.283617 (+ 1849us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 10:17:31.283618 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 10:17:31.283618 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 10:17:31.283625 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 10:17:31.284805 (+ 1180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:17:31.285157 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:17:31.285157 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:17:31.285158 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:17:31.285209 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:17:31.285393 (+ 184us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:17:31.285393 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:17:31.285394 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:17:31.285490 (+ 96us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 10:17:31.285562 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0619 10:17:31.285722 (+ 160us) server_negotiation.cc:299] Negotiation successful
0619 10:17:31.285791 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260619 10:17:31.285912 7456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:31.279579 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.125:40043 (local address 127.0.0.1:38808)
0619 10:17:31.279706 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0619 10:17:31.279734 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0619 10:17:31.279812 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 10:17:31.279957 (+ 145us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 10:17:31.279960 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 10:17:31.280199 (+ 239us) client_negotiation.cc:189] Negotiated authn=SASL
0619 10:17:31.280236 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:31.280237 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:31.280847 (+ 610us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:31.280848 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:31.281417 (+ 569us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:31.281419 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:31.281772 (+ 353us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:31.281773 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:31.281838 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:31.282269 (+ 431us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 10:17:31.282273 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 10:17:31.283537 (+ 1264us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 10:17:31.284880 (+ 1343us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 10:17:31.284886 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 10:17:31.284888 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 10:17:31.285076 (+ 188us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 10:17:31.285270 (+ 194us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 10:17:31.285270 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 10:17:31.285271 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 10:17:31.285325 (+ 54us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 10:17:31.285557 (+ 232us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 10:17:31.285563 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 10:17:31.285609 (+ 46us) client_negotiation.cc:769] Sending connection context
0619 10:17:31.285699 (+ 90us) client_negotiation.cc:240] Negotiation successful
0619 10:17:31.285801 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":58,"threads_started":1}
I20260619 10:17:31.286345 6898 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.0.206.125:40043: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "cd27214b90874d35b38271d228518b20"
member_type: NON_VOTER
last_known_addr {
host: "127.0.206.125"
port: 40043
}
attrs {
promote: true
}
}
dest_uuid: "bd146d3a022940a68274ddac0993c2e3"
cas_config_opid_index: -1
I20260619 10:17:31.286654 6898 consensus_queue.cc:237] T 00000000000000000000000000000000 P bd146d3a022940a68274ddac0993c2e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "bd146d3a022940a68274ddac0993c2e3" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 38663 } } peers { permanent_uuid: "cd27214b90874d35b38271d228518b20" member_type: NON_VOTER last_known_addr { host: "127.0.206.125" port: 40043 } attrs { promote: true } }
I20260619 10:17:31.287089 7458 raft_consensus.cc:2932] T 00000000000000000000000000000000 P bd146d3a022940a68274ddac0993c2e3 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER cd27214b90874d35b38271d228518b20 (127.0.206.125) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "bd146d3a022940a68274ddac0993c2e3" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 38663 } } peers { permanent_uuid: "cd27214b90874d35b38271d228518b20" member_type: NON_VOTER last_known_addr { host: "127.0.206.125" port: 40043 } attrs { promote: true } } }
I20260619 10:17:31.287309 7404 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cd27214b90874d35b38271d228518b20 [term 0 FOLLOWER]: Advancing to term 1
I20260619 10:17:31.288154 7404 raft_consensus.cc:1261] T 00000000000000000000000000000000 P cd27214b90874d35b38271d228518b20 [term 1 FOLLOWER]: Refusing update from remote peer bd146d3a022940a68274ddac0993c2e3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260619 10:17:31.288188 7458 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.0.206.125:40043
I20260619 10:17:31.288252 7458 sys_catalog.cc:455] T 00000000000000000000000000000000 P bd146d3a022940a68274ddac0993c2e3 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "bd146d3a022940a68274ddac0993c2e3" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "bd146d3a022940a68274ddac0993c2e3" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 38663 } } peers { permanent_uuid: "cd27214b90874d35b38271d228518b20" member_type: NON_VOTER last_known_addr { host: "127.0.206.125" port: 40043 } attrs { promote: true } } }
I20260619 10:17:31.288316 7458 sys_catalog.cc:458] T 00000000000000000000000000000000 P bd146d3a022940a68274ddac0993c2e3 [sys.catalog]: This master's current role is: LEADER
I20260619 10:17:31.288417 7458 consensus_queue.cc:1035] T 00000000000000000000000000000000 P bd146d3a022940a68274ddac0993c2e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd27214b90874d35b38271d228518b20" member_type: NON_VOTER last_known_addr { host: "127.0.206.125" port: 40043 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 10:17:31.288443 7362 master.cc:537] Master@127.0.206.125:40043 shutting down...
W20260619 10:17:31.289007 6884 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
W20260619 10:17:31.289388 6884 consensus_peers.cc:487] T 00000000000000000000000000000000 P bd146d3a022940a68274ddac0993c2e3 -> Peer cd27214b90874d35b38271d228518b20 (127.0.206.125:40043): Couldn't send request to peer cd27214b90874d35b38271d228518b20. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260619 10:17:31.291451 7362 raft_consensus.cc:2229] T 00000000000000000000000000000000 P cd27214b90874d35b38271d228518b20 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 10:17:31.291538 7362 raft_consensus.cc:2258] T 00000000000000000000000000000000 P cd27214b90874d35b38271d228518b20 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 10:17:31.291587 7362 tablet_replica.cc:330] T 00000000000000000000000000000000 P cd27214b90874d35b38271d228518b20: stopping tablet replica
I20260619 10:17:31.303609 7362 master.cc:552] Master@127.0.206.125:40043 shutdown complete.
I20260619 10:17:31.303645 7362 master_runner.cc:305] Clearing existing system tablet
I20260619 10:17:31.303773 7362 fs_manager.cc:359] Metadata directory not provided
I20260619 10:17:31.303809 7362 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 10:17:31.304389 7362 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.000s sys 0.001s
I20260619 10:17:31.304790 7463 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 10:17:31.304903 7362 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781864224577058-825-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 10:17:31.304955 7362 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260619 10:17:31.304976 7362 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781864224577058-825-0/minicluster-data/master-1/data,/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781864224577058-825-0/minicluster-data/master-1/wal
uuid: "cd27214b90874d35b38271d228518b20"
format_stamp: "Formatted at 2026-06-19 10:17:31 on dist-test-slave-6pt2"
server_key: "83b12de1595d168f62089f11371cdded"
server_key_iv: "0713785e593d35c7409d1dda47e86324"
server_key_version: "encryptionkey@0"
I20260619 10:17:31.305541 7362 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P cd27214b90874d35b38271d228518b20: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 10:17:31.306733 7362 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P cd27214b90874d35b38271d228518b20: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260619 10:17:31.306783 7362 log.cc:1196] T 00000000000000000000000000000000 P cd27214b90874d35b38271d228518b20: Deleting WAL directory at /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781864224577058-825-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260619 10:17:31.306964 7362 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P cd27214b90874d35b38271d228518b20: Deleting consensus metadata
I20260619 10:17:31.307096 7362 master_runner.cc:315] Copying system tablet from 127.0.206.126:38663
I20260619 10:17:31.307649 7362 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P cd27214b90874d35b38271d228518b20: tablet copy: Beginning tablet copy session from remote peer at address 127.0.206.126:38663
I20260619 10:17:31.313730 7469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:31.307924 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:38663 (local address 127.0.0.1:60202)
0619 10:17:31.308031 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0619 10:17:31.308042 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 10:17:31.308092 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 10:17:31.308220 (+ 128us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 10:17:31.308221 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 10:17:31.308353 (+ 132us) client_negotiation.cc:189] Negotiated authn=SASL
0619 10:17:31.308619 (+ 266us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:31.308620 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:31.309493 (+ 873us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:31.309493 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:31.310037 (+ 544us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:31.310039 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:31.310493 (+ 454us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:31.310493 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:31.310543 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:31.311019 (+ 476us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 10:17:31.311023 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 10:17:31.311297 (+ 274us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 10:17:31.312864 (+ 1567us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 10:17:31.312865 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 10:17:31.312865 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 10:17:31.313028 (+ 163us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 10:17:31.313216 (+ 188us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 10:17:31.313217 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 10:17:31.313217 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 10:17:31.313263 (+ 46us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 10:17:31.313511 (+ 248us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 10:17:31.313511 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 10:17:31.313543 (+ 32us) client_negotiation.cc:769] Sending connection context
0619 10:17:31.313595 (+ 52us) client_negotiation.cc:240] Negotiation successful
0619 10:17:31.313662 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":82,"thread_start_us":59,"threads_started":1}
I20260619 10:17:31.313792 6956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:31.308065 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60202 (local address 127.0.206.126:38663)
0619 10:17:31.308112 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0619 10:17:31.308114 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 10:17:31.308123 (+ 9us) server_negotiation.cc:407] Connection header received
0619 10:17:31.308144 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:17:31.308145 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:17:31.308154 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:17:31.308217 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:17:31.308688 (+ 471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:31.309423 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:31.310116 (+ 693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:31.310414 (+ 298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:31.310481 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:31.311367 (+ 886us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 10:17:31.311368 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 10:17:31.311369 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 10:17:31.311375 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 10:17:31.312792 (+ 1417us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:17:31.313091 (+ 299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:17:31.313092 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:17:31.313092 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:17:31.313156 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:17:31.313314 (+ 158us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:17:31.313315 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:17:31.313316 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:17:31.313447 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 10:17:31.313506 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0619 10:17:31.313630 (+ 124us) server_negotiation.cc:299] Negotiation successful
0619 10:17:31.313701 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260619 10:17:31.314021 6918 tablet_copy_service.cc:140] P bd146d3a022940a68274ddac0993c2e3: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer cd27214b90874d35b38271d228518b20 ({username='oryx', principal='oryx/127.0.206.125@KRBTEST.COM'} at 127.0.0.1:60202)
I20260619 10:17:31.314111 6918 tablet_copy_service.cc:161] P bd146d3a022940a68274ddac0993c2e3: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer cd27214b90874d35b38271d228518b20 at {username='oryx', principal='oryx/127.0.206.125@KRBTEST.COM'} at 127.0.0.1:60202: session id = cd27214b90874d35b38271d228518b20-00000000000000000000000000000000
I20260619 10:17:31.314734 6918 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P bd146d3a022940a68274ddac0993c2e3: Tablet Copy: opened 0 blocks and 1 log segments
I20260619 10:17:31.315275 7362 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260619 10:17:31.316391 7362 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P cd27214b90874d35b38271d228518b20: tablet copy: Starting download of 0 data blocks...
I20260619 10:17:31.316529 7362 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P cd27214b90874d35b38271d228518b20: tablet copy: Starting download of 1 WAL segments...
I20260619 10:17:31.317299 7362 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P cd27214b90874d35b38271d228518b20: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260619 10:17:31.318127 6918 tablet_copy_service.cc:342] P bd146d3a022940a68274ddac0993c2e3: Request end of tablet copy session cd27214b90874d35b38271d228518b20-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.0.206.125@KRBTEST.COM'} at 127.0.0.1:60202
I20260619 10:17:31.318192 6918 tablet_copy_service.cc:433] P bd146d3a022940a68274ddac0993c2e3: ending tablet copy session cd27214b90874d35b38271d228518b20-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer cd27214b90874d35b38271d228518b20
I20260619 10:17:31.320225 7362 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 10:17:31.321259 7470 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 10:17:31.321329 7473 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 10:17:31.321310 7471 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 10:17:31.321455 7362 server_base.cc:1034] running on GCE node
I20260619 10:17:31.321575 7362 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 10:17:31.321704 7362 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 10:17:31.322816 7362 hybrid_clock.cc:648] HybridClock initialized: now 1781864251322808 us; error 28 us; skew 500 ppm
Jun 19 10:17:31 dist-test-slave-6pt2 krb5kdc[6853](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864251, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.206.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 10:17:31.324095 7362 init.cc:377] Logged in from keytab as oryx/127.0.206.125@KRBTEST.COM (short username oryx)
I20260619 10:17:31.324337 7362 webserver.cc:460] Webserver started at http://127.0.206.125:35709/ using document root <none> and password file <none>
I20260619 10:17:31.324477 7362 fs_manager.cc:359] Metadata directory not provided
I20260619 10:17:31.324519 7362 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 10:17:31.325140 7362 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.001s sys 0.000s
I20260619 10:17:31.325582 7480 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 10:17:31.325702 7362 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260619 10:17:31.325764 7362 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781864224577058-825-0/minicluster-data/master-1/data,/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781864224577058-825-0/minicluster-data/master-1/wal
uuid: "cd27214b90874d35b38271d228518b20"
format_stamp: "Formatted at 2026-06-19 10:17:31 on dist-test-slave-6pt2"
server_key: "83b12de1595d168f62089f11371cdded"
server_key_iv: "0713785e593d35c7409d1dda47e86324"
server_key_version: "encryptionkey@0"
I20260619 10:17:31.325812 7362 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781864224577058-825-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781864224577058-825-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781864224577058-825-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 10:17:31.335002 7362 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 10:17:31.336408 7362 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 10:17:31.339203 7362 rpc_server.cc:307] RPC server started. Bound to: 127.0.206.125:40043
I20260619 10:17:31.339507 7532 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.206.125:40043 every 8 connection(s)
I20260619 10:17:31.339560 7362 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781864224577058-825-0/minicluster-data/master-1/data/info.pb
I20260619 10:17:31.340270 7533 sys_catalog.cc:263] Verifying existing consensus state
I20260619 10:17:31.340646 7533 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cd27214b90874d35b38271d228518b20: Bootstrap starting.
I20260619 10:17:31.342888 7533 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cd27214b90874d35b38271d228518b20: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260619 10:17:31.343112 7533 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cd27214b90874d35b38271d228518b20: Bootstrap complete.
I20260619 10:17:31.343281 7533 raft_consensus.cc:348] T 00000000000000000000000000000000 P cd27214b90874d35b38271d228518b20 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "bd146d3a022940a68274ddac0993c2e3" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 38663 } } peers { permanent_uuid: "cd27214b90874d35b38271d228518b20" member_type: NON_VOTER last_known_addr { host: "127.0.206.125" port: 40043 } attrs { promote: true } }
I20260619 10:17:31.343354 7533 raft_consensus.cc:729] T 00000000000000000000000000000000 P cd27214b90874d35b38271d228518b20 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: cd27214b90874d35b38271d228518b20, State: Initialized, Role: LEARNER
I20260619 10:17:31.343395 7533 consensus_queue.cc:260] T 00000000000000000000000000000000 P cd27214b90874d35b38271d228518b20 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "bd146d3a022940a68274ddac0993c2e3" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 38663 } } peers { permanent_uuid: "cd27214b90874d35b38271d228518b20" member_type: NON_VOTER last_known_addr { host: "127.0.206.125" port: 40043 } attrs { promote: true } }
I20260619 10:17:31.343550 7533 sys_catalog.cc:564] T 00000000000000000000000000000000 P cd27214b90874d35b38271d228518b20 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 10:17:31.343550 7536 sys_catalog.cc:455] T 00000000000000000000000000000000 P cd27214b90874d35b38271d228518b20 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "bd146d3a022940a68274ddac0993c2e3" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 38663 } } peers { permanent_uuid: "cd27214b90874d35b38271d228518b20" member_type: NON_VOTER last_known_addr { host: "127.0.206.125" port: 40043 } attrs { promote: true } } }
I20260619 10:17:31.343636 7536 sys_catalog.cc:458] T 00000000000000000000000000000000 P cd27214b90874d35b38271d228518b20 [sys.catalog]: This master's current role is: LEARNER
I20260619 10:17:31.344928 7547 catalog_manager.cc:1259] Loaded cluster ID: a825fc644b564a9897f7dc902ab7b888
I20260619 10:17:31.344986 7547 catalog_manager.cc:1552] T 00000000000000000000000000000000 P cd27214b90874d35b38271d228518b20: loading cluster ID for follower catalog manager: success
I20260619 10:17:31.345804 7547 catalog_manager.cc:1574] T 00000000000000000000000000000000 P cd27214b90874d35b38271d228518b20: acquiring CA information for follower catalog manager: success
I20260619 10:17:31.346050 7547 catalog_manager.cc:1602] T 00000000000000000000000000000000 P cd27214b90874d35b38271d228518b20: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260619 10:17:31.364980 7548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:31.358537 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38826 (local address 127.0.206.125:40043)
0619 10:17:31.358683 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0619 10:17:31.358685 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 10:17:31.358695 (+ 10us) server_negotiation.cc:407] Connection header received
0619 10:17:31.358718 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:17:31.358719 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:17:31.358730 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:17:31.358815 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:17:31.359297 (+ 482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:31.360320 (+ 1023us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:31.360982 (+ 662us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:31.361289 (+ 307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:31.361371 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:31.362437 (+ 1066us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 10:17:31.362438 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 10:17:31.362439 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 10:17:31.362445 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 10:17:31.363956 (+ 1511us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:17:31.364261 (+ 305us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:17:31.364263 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:17:31.364263 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:17:31.364325 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:17:31.364495 (+ 170us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:17:31.364496 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:17:31.364496 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:17:31.364625 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 10:17:31.364726 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0619 10:17:31.364803 (+ 77us) server_negotiation.cc:299] Negotiation successful
0619 10:17:31.364878 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":120,"thread_start_us":83,"threads_started":1}
WARNING: no policy specified for oryx/127.0.206.124@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.0.206.124@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.0.206.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781864224577058-825-0/krb5kdc/oryx_127.0.206.124.keytab.
Entry for principal oryx/127.0.206.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781864224577058-825-0/krb5kdc/oryx_127.0.206.124.keytab.
I20260619 10:17:31.388826 825 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTU9OP6/build/release/bin/kudu
/tmp/dist-test-taskTU9OP6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781864224577058-825-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781864224577058-825-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781864224577058-825-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781864224577058-825-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.206.124:42647
--webserver_interface=127.0.206.124
--webserver_port=0
--keytab_file=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781864224577058-825-0/krb5kdc/oryx_127.0.206.124.keytab
--principal=oryx/127.0.206.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.206.84:33273
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.206.126:38663,127.0.206.125:40043,127.0.206.124:42647
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781864224577058-825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781864224577058-825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781864224577058-825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 10:17:31.468390 7551 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 10:17:31.468611 7551 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 10:17:31.468631 7551 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 10:17:31.469727 7551 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 10:17:31.469767 7551 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 10:17:31.469782 7551 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 10:17:31.469795 7551 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 10:17:31.469872 7551 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 10:17:31.471424 7551 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.206.84:33273
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781864224577058-825-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781864224577058-825-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.0.206.126:38663,127.0.206.125:40043,127.0.206.124:42647
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781864224577058-825-0/krb5kdc/oryx_127.0.206.124.keytab
--principal=oryx/127.0.206.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.206.124:42647
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781864224577058-825-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.0.206.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7551
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781864224577058-825-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 09:32:44 UTC on ccycloud.quasar-gbtldp.root.comops.site
build id 4455440
I20260619 10:17:31.471675 7551 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 10:17:31.471900 7551 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 10:17:31.476466 7557 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 10:17:31.476486 7556 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 10:17:31.476486 7559 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 10:17:31.476706 7551 server_base.cc:1034] running on GCE node
I20260619 10:17:31.476930 7551 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 10:17:31.477231 7551 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 10:17:31.478376 7551 hybrid_clock.cc:648] HybridClock initialized: now 1781864251478368 us; error 32 us; skew 500 ppm
Jun 19 10:17:31 dist-test-slave-6pt2 krb5kdc[6853](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864251, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.206.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 10:17:31.481235 7551 init.cc:377] Logged in from keytab as oryx/127.0.206.124@KRBTEST.COM (short username oryx)
I20260619 10:17:31.481679 7551 webserver.cc:460] Webserver started at http://127.0.206.124:36489/ using document root <none> and password file <none>
I20260619 10:17:31.481914 7551 fs_manager.cc:359] Metadata directory not provided
I20260619 10:17:31.481957 7551 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 10:17:31.482051 7551 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 10:17:31.483183 7551 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781864224577058-825-0/minicluster-data/master-2/data/instance:
uuid: "4bac10aeb06347f2bfadf295a3417d44"
format_stamp: "Formatted at 2026-06-19 10:17:31 on dist-test-slave-6pt2"
server_key: "63d6bacf0174d66832d11af48c1bc1f4"
server_key_iv: "bc3333f62bdbbc90389129e6ff596439"
server_key_version: "encryptionkey@0"
I20260619 10:17:31.483465 7551 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781864224577058-825-0/minicluster-data/master-2/wal/instance:
uuid: "4bac10aeb06347f2bfadf295a3417d44"
format_stamp: "Formatted at 2026-06-19 10:17:31 on dist-test-slave-6pt2"
server_key: "63d6bacf0174d66832d11af48c1bc1f4"
server_key_iv: "bc3333f62bdbbc90389129e6ff596439"
server_key_version: "encryptionkey@0"
I20260619 10:17:31.484632 7551 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 10:17:31.485224 7566 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 10:17:31.485386 7551 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260619 10:17:31.485461 7551 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781864224577058-825-0/minicluster-data/master-2/data,/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781864224577058-825-0/minicluster-data/master-2/wal
uuid: "4bac10aeb06347f2bfadf295a3417d44"
format_stamp: "Formatted at 2026-06-19 10:17:31 on dist-test-slave-6pt2"
server_key: "63d6bacf0174d66832d11af48c1bc1f4"
server_key_iv: "bc3333f62bdbbc90389129e6ff596439"
server_key_version: "encryptionkey@0"
I20260619 10:17:31.485513 7551 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781864224577058-825-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781864224577058-825-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781864224577058-825-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 10:17:31.507623 7551 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 10:17:31.509296 7551 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 10:17:31.509402 7551 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 10:17:31.513537 7551 rpc_server.cc:307] RPC server started. Bound to: 127.0.206.124:42647
I20260619 10:17:31.513589 7618 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.206.124:42647 every 8 connection(s)
I20260619 10:17:31.513938 7551 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781864224577058-825-0/minicluster-data/master-2/data/info.pb
I20260619 10:17:31.514941 7619 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 10:17:31.516408 7619 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.206.126" port: 38663 } has no permanent_uuid. Determining permanent_uuid...
Jun 19 10:17:31 dist-test-slave-6pt2 krb5kdc[6853](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864251, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.206.124@KRBTEST.COM for oryx/127.0.206.126@KRBTEST.COM
I20260619 10:17:31.523332 825 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTU9OP6/build/release/bin/kudu as pid 7551
I20260619 10:17:31.524515 6956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:31.517150 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37412 (local address 127.0.206.126:38663)
0619 10:17:31.517205 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0619 10:17:31.517206 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 10:17:31.517330 (+ 124us) server_negotiation.cc:407] Connection header received
0619 10:17:31.517773 (+ 443us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:17:31.517774 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:17:31.517782 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:17:31.517825 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:17:31.518345 (+ 520us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:31.518907 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:31.519826 (+ 919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:31.520001 (+ 175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:31.520049 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:31.521902 (+ 1853us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 10:17:31.521903 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 10:17:31.521903 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 10:17:31.521910 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 10:17:31.523100 (+ 1190us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:17:31.523513 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:17:31.523515 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:17:31.523515 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:17:31.523578 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:17:31.523846 (+ 268us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:17:31.523847 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:17:31.523848 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:17:31.523989 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 10:17:31.524062 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0619 10:17:31.524355 (+ 293us) server_negotiation.cc:299] Negotiation successful
0619 10:17:31.524430 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260619 10:17:31.524694 7621 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:31.517014 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:38663 (local address 127.0.0.1:37412)
0619 10:17:31.517235 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0619 10:17:31.517265 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0619 10:17:31.517667 (+ 402us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 10:17:31.517889 (+ 222us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 10:17:31.517892 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 10:17:31.518114 (+ 222us) client_negotiation.cc:189] Negotiated authn=SASL
0619 10:17:31.518281 (+ 167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:31.518285 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:31.518983 (+ 698us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:31.518984 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:31.519760 (+ 776us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:31.519762 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:31.520062 (+ 300us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:31.520062 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:31.520154 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:31.520592 (+ 438us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 10:17:31.520599 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0619 10:17:31.521825 (+ 1226us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 10:17:31.523187 (+ 1362us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 10:17:31.523192 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 10:17:31.523194 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 10:17:31.523420 (+ 226us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 10:17:31.523642 (+ 222us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 10:17:31.523643 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 10:17:31.523643 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 10:17:31.523738 (+ 95us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 10:17:31.524056 (+ 318us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 10:17:31.524061 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 10:17:31.524188 (+ 127us) client_negotiation.cc:769] Sending connection context
0619 10:17:31.524319 (+ 131us) client_negotiation.cc:240] Negotiation successful
0619 10:17:31.524465 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":178,"thread_start_us":79,"threads_started":1}
I20260619 10:17:31.525203 7619 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.206.125" port: 40043 } has no permanent_uuid. Determining permanent_uuid...
Jun 19 10:17:31 dist-test-slave-6pt2 krb5kdc[6853](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864250, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.0.206.124@KRBTEST.COM
Jun 19 10:17:31 dist-test-slave-6pt2 krb5kdc[6853](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864251, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.206.124@KRBTEST.COM for oryx/127.0.206.125@KRBTEST.COM
I20260619 10:17:31.532639 7548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:31.525527 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48768 (local address 127.0.206.125:40043)
0619 10:17:31.525577 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0619 10:17:31.525578 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 10:17:31.525586 (+ 8us) server_negotiation.cc:407] Connection header received
0619 10:17:31.525610 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:17:31.525611 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:17:31.525626 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:17:31.525692 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:17:31.525938 (+ 246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:31.526719 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:31.527564 (+ 845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:31.527788 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:31.527856 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:31.530059 (+ 2203us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 10:17:31.530060 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 10:17:31.530060 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 10:17:31.530067 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 10:17:31.531547 (+ 1480us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:17:31.531860 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:17:31.531861 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:17:31.531862 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:17:31.531914 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:17:31.532112 (+ 198us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:17:31.532113 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:17:31.532113 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:17:31.532228 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 10:17:31.532300 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0619 10:17:31.532441 (+ 141us) server_negotiation.cc:299] Negotiation successful
0619 10:17:31.532526 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260619 10:17:31.532634 7622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:31.523799 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42020 (local address 127.0.206.124:42647)
0619 10:17:31.523943 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0619 10:17:31.523947 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 10:17:31.523967 (+ 20us) server_negotiation.cc:407] Connection header received
0619 10:17:31.523983 (+ 16us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:17:31.523984 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:17:31.523994 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:17:31.524072 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:17:31.524647 (+ 575us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:31.525636 (+ 989us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:31.526676 (+ 1040us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:31.527134 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:31.527197 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:31.529625 (+ 2428us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 10:17:31.529631 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 10:17:31.529631 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 10:17:31.529638 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 10:17:31.531402 (+ 1764us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:17:31.531759 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:17:31.531762 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:17:31.531763 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:17:31.531820 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:17:31.532010 (+ 190us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:17:31.532011 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:17:31.532012 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:17:31.532140 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 10:17:31.532274 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0619 10:17:31.532394 (+ 120us) server_negotiation.cc:299] Negotiation successful
0619 10:17:31.532477 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":6,"server-negotiator.queue_time_us":117,"thread_start_us":96,"threads_started":1}
I20260619 10:17:31.532709 7621 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:31.525451 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.125:40043 (local address 127.0.0.1:48768)
0619 10:17:31.525492 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0619 10:17:31.525502 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 10:17:31.525542 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 10:17:31.525695 (+ 153us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 10:17:31.525696 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 10:17:31.525837 (+ 141us) client_negotiation.cc:189] Negotiated authn=SASL
0619 10:17:31.525875 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:31.525876 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:31.526790 (+ 914us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:31.526791 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:31.527472 (+ 681us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:31.527473 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:31.527865 (+ 392us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:31.527866 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:31.527935 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:31.528292 (+ 357us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 10:17:31.528295 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0619 10:17:31.529977 (+ 1682us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 10:17:31.531612 (+ 1635us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 10:17:31.531613 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 10:17:31.531614 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 10:17:31.531777 (+ 163us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 10:17:31.531990 (+ 213us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 10:17:31.531990 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 10:17:31.531991 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 10:17:31.532048 (+ 57us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 10:17:31.532293 (+ 245us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 10:17:31.532293 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 10:17:31.532332 (+ 39us) client_negotiation.cc:769] Sending connection context
0619 10:17:31.532415 (+ 83us) client_negotiation.cc:240] Negotiation successful
0619 10:17:31.532528 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":27,"mutex_wait_us":29}
I20260619 10:17:31.533150 7619 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.206.124" port: 42647 } has no permanent_uuid. Determining permanent_uuid...
Jun 19 10:17:31 dist-test-slave-6pt2 krb5kdc[6853](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864251, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.206.124@KRBTEST.COM for oryx/127.0.206.124@KRBTEST.COM
I20260619 10:17:31.538278 7621 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:31.533354 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.124:42647 (local address 127.0.0.1:42034)
0619 10:17:31.533410 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0619 10:17:31.533420 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 10:17:31.533457 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 10:17:31.533549 (+ 92us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 10:17:31.533550 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 10:17:31.533648 (+ 98us) client_negotiation.cc:189] Negotiated authn=SASL
0619 10:17:31.533670 (+ 22us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:31.533671 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:31.534174 (+ 503us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:31.534175 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:31.534645 (+ 470us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:31.534646 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:31.534952 (+ 306us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:31.534953 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:31.534985 (+ 32us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:31.535277 (+ 292us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 10:17:31.535281 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 10:17:31.536227 (+ 946us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 10:17:31.537635 (+ 1408us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 10:17:31.537635 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 10:17:31.537636 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 10:17:31.537768 (+ 132us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 10:17:31.537899 (+ 131us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 10:17:31.537900 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 10:17:31.537900 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 10:17:31.537943 (+ 43us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 10:17:31.538092 (+ 149us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 10:17:31.538093 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 10:17:31.538120 (+ 27us) client_negotiation.cc:769] Sending connection context
0619 10:17:31.538166 (+ 46us) client_negotiation.cc:240] Negotiation successful
0619 10:17:31.538227 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":42}
I20260619 10:17:31.538334 7622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:31.533421 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42034 (local address 127.0.206.124:42647)
0619 10:17:31.533453 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0619 10:17:31.533455 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 10:17:31.533468 (+ 13us) server_negotiation.cc:407] Connection header received
0619 10:17:31.533501 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:17:31.533502 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:17:31.533508 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:17:31.533548 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:17:31.533707 (+ 159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:31.534121 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:31.534723 (+ 602us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:31.534898 (+ 175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:31.534947 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:31.536294 (+ 1347us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 10:17:31.536295 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 10:17:31.536295 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 10:17:31.536301 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 10:17:31.537574 (+ 1273us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:17:31.537817 (+ 243us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:17:31.537817 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:17:31.537817 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:17:31.537855 (+ 38us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:17:31.537977 (+ 122us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:17:31.537977 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:17:31.537977 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:17:31.538050 (+ 73us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 10:17:31.538091 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0619 10:17:31.538181 (+ 90us) server_negotiation.cc:299] Negotiation successful
0619 10:17:31.538266 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":35,"server-negotiator.queue_time_us":17}
I20260619 10:17:31.539453 7619 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4bac10aeb06347f2bfadf295a3417d44: Bootstrap starting.
I20260619 10:17:31.540179 7619 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4bac10aeb06347f2bfadf295a3417d44: Neither blocks nor log segments found. Creating new log.
I20260619 10:17:31.540413 7619 log.cc:824] T 00000000000000000000000000000000 P 4bac10aeb06347f2bfadf295a3417d44: Log is configured to *not* fsync() on all Append() calls
I20260619 10:17:31.540961 7619 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4bac10aeb06347f2bfadf295a3417d44: No bootstrap required, opened a new log
I20260619 10:17:31.542256 7619 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4bac10aeb06347f2bfadf295a3417d44 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd146d3a022940a68274ddac0993c2e3" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 38663 } } peers { permanent_uuid: "cd27214b90874d35b38271d228518b20" member_type: VOTER last_known_addr { host: "127.0.206.125" port: 40043 } } peers { permanent_uuid: "4bac10aeb06347f2bfadf295a3417d44" member_type: VOTER last_known_addr { host: "127.0.206.124" port: 42647 } }
I20260619 10:17:31.542361 7619 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4bac10aeb06347f2bfadf295a3417d44 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 10:17:31.542382 7619 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4bac10aeb06347f2bfadf295a3417d44 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4bac10aeb06347f2bfadf295a3417d44, State: Initialized, Role: FOLLOWER
I20260619 10:17:31.542646 7619 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4bac10aeb06347f2bfadf295a3417d44 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd146d3a022940a68274ddac0993c2e3" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 38663 } } peers { permanent_uuid: "cd27214b90874d35b38271d228518b20" member_type: VOTER last_known_addr { host: "127.0.206.125" port: 40043 } } peers { permanent_uuid: "4bac10aeb06347f2bfadf295a3417d44" member_type: VOTER last_known_addr { host: "127.0.206.124" port: 42647 } }
I20260619 10:17:31.542883 7619 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4bac10aeb06347f2bfadf295a3417d44 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 10:17:31.542874 7624 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4bac10aeb06347f2bfadf295a3417d44 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd146d3a022940a68274ddac0993c2e3" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 38663 } } peers { permanent_uuid: "cd27214b90874d35b38271d228518b20" member_type: VOTER last_known_addr { host: "127.0.206.125" port: 40043 } } peers { permanent_uuid: "4bac10aeb06347f2bfadf295a3417d44" member_type: VOTER last_known_addr { host: "127.0.206.124" port: 42647 } } }
I20260619 10:17:31.542979 7624 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4bac10aeb06347f2bfadf295a3417d44 [sys.catalog]: This master's current role is: FOLLOWER
W20260619 10:17:31.544572 7635 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 4bac10aeb06347f2bfadf295a3417d44: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 10:17:31.544638 7635 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 10:17:31.546473 7551 master_runner.cc:428] Detected that this master 4bac10aeb06347f2bfadf295a3417d44 is joining an existing cluster
I20260619 10:17:31.546520 7551 master_runner.cc:432] Initiating AddMaster RPC to add 127.0.206.124:42647
I20260619 10:17:31.553925 7548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:31.547705 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48774 (local address 127.0.206.125:40043)
0619 10:17:31.547758 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0619 10:17:31.547760 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 10:17:31.547956 (+ 196us) server_negotiation.cc:407] Connection header received
0619 10:17:31.548020 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:17:31.548021 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:17:31.548029 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:17:31.548080 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:17:31.548724 (+ 644us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:31.549457 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:31.550406 (+ 949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:31.550652 (+ 246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:31.550715 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:31.551636 (+ 921us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 10:17:31.551637 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 10:17:31.551638 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 10:17:31.551644 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 10:17:31.552870 (+ 1226us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:17:31.553194 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:17:31.553195 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:17:31.553195 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:17:31.553246 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:17:31.553435 (+ 189us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:17:31.553436 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:17:31.553436 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:17:31.553541 (+ 105us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 10:17:31.553609 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0619 10:17:31.553763 (+ 154us) server_negotiation.cc:299] Negotiation successful
0619 10:17:31.553831 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260619 10:17:31.554111 7643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:31.547660 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.125:40043 (local address 127.0.0.1:48774)
0619 10:17:31.547900 (+ 240us) negotiation.cc:107] Waiting for socket to connect
0619 10:17:31.547909 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0619 10:17:31.547961 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 10:17:31.548081 (+ 120us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 10:17:31.548081 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 10:17:31.548297 (+ 216us) client_negotiation.cc:189] Negotiated authn=SASL
0619 10:17:31.548370 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:31.548372 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:31.549536 (+ 1164us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:31.549537 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:31.550328 (+ 791us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:31.550329 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:31.550764 (+ 435us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:31.550765 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:31.550841 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:31.551260 (+ 419us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 10:17:31.551264 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 10:17:31.551554 (+ 290us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 10:17:31.552945 (+ 1391us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 10:17:31.552946 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 10:17:31.552946 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 10:17:31.553114 (+ 168us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 10:17:31.553310 (+ 196us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 10:17:31.553311 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 10:17:31.553311 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 10:17:31.553365 (+ 54us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 10:17:31.553601 (+ 236us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 10:17:31.553602 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 10:17:31.553663 (+ 61us) client_negotiation.cc:769] Sending connection context
0619 10:17:31.553740 (+ 77us) client_negotiation.cc:240] Negotiation successful
0619 10:17:31.553831 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":226,"mutex_wait_us":6,"thread_start_us":60,"threads_started":1}
I20260619 10:17:31.574321 6956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:31.547520 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37422 (local address 127.0.206.126:38663)
0619 10:17:31.547570 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0619 10:17:31.547573 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 10:17:31.547712 (+ 139us) server_negotiation.cc:407] Connection header received
0619 10:17:31.547948 (+ 236us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:17:31.547950 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:17:31.547962 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:17:31.548012 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:17:31.548812 (+ 800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:31.549271 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:31.550274 (+ 1003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:31.550573 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:31.550684 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:31.571496 (+ 20812us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 10:17:31.571498 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 10:17:31.571498 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 10:17:31.571505 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 10:17:31.573105 (+ 1600us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:17:31.573505 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:17:31.573506 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:17:31.573507 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:17:31.573568 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:17:31.573753 (+ 185us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:17:31.573754 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:17:31.573754 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:17:31.573884 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 10:17:31.573953 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0619 10:17:31.574145 (+ 192us) server_negotiation.cc:299] Negotiation successful
0619 10:17:31.574225 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260619 10:17:31.574326 7641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:31.547448 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:38663 (local address 127.0.0.1:37422)
0619 10:17:31.547636 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0619 10:17:31.547648 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 10:17:31.547787 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 10:17:31.548053 (+ 266us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 10:17:31.548054 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 10:17:31.548293 (+ 239us) client_negotiation.cc:189] Negotiated authn=SASL
0619 10:17:31.548744 (+ 451us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:31.548745 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:31.549345 (+ 600us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:31.549346 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:31.550187 (+ 841us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:31.550189 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:31.550748 (+ 559us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:31.550749 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:31.550818 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:31.571093 (+ 20275us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 10:17:31.571101 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 10:17:31.571411 (+ 310us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 10:17:31.573191 (+ 1780us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 10:17:31.573192 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 10:17:31.573192 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 10:17:31.573416 (+ 224us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 10:17:31.573630 (+ 214us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 10:17:31.573631 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 10:17:31.573631 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 10:17:31.573687 (+ 56us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 10:17:31.573954 (+ 267us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 10:17:31.573955 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 10:17:31.573998 (+ 43us) client_negotiation.cc:769] Sending connection context
0619 10:17:31.574110 (+ 112us) client_negotiation.cc:240] Negotiation successful
0619 10:17:31.574216 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":166,"thread_start_us":98,"threads_started":1}
I20260619 10:17:31.577008 7622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:31.547864 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42036 (local address 127.0.206.124:42647)
0619 10:17:31.547901 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0619 10:17:31.547903 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 10:17:31.550745 (+ 2842us) server_negotiation.cc:407] Connection header received
0619 10:17:31.550821 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:17:31.550822 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:17:31.550830 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:17:31.550855 (+ 25us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:17:31.570977 (+ 20122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:31.571744 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:31.572697 (+ 953us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:31.572875 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:31.572948 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:31.573844 (+ 896us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 10:17:31.573845 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 10:17:31.573846 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 10:17:31.573851 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0619 10:17:31.575678 (+ 1827us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:17:31.576260 (+ 582us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:17:31.576261 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:17:31.576262 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:17:31.576317 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:17:31.576514 (+ 197us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:17:31.576515 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:17:31.576515 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:17:31.576643 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 10:17:31.576685 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0619 10:17:31.576864 (+ 179us) server_negotiation.cc:299] Negotiation successful
0619 10:17:31.576937 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260619 10:17:31.577263 7642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:31.547808 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.124:42647 (local address 127.0.0.1:42036)
0619 10:17:31.550691 (+ 2883us) negotiation.cc:107] Waiting for socket to connect
0619 10:17:31.550702 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 10:17:31.550764 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 10:17:31.550901 (+ 137us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 10:17:31.550902 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 10:17:31.570807 (+ 19905us) client_negotiation.cc:189] Negotiated authn=SASL
0619 10:17:31.570864 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:31.570866 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:31.571826 (+ 960us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:31.571827 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:31.572608 (+ 781us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:31.572610 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:31.572949 (+ 339us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:31.572950 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:31.573007 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:31.573386 (+ 379us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 10:17:31.573392 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0619 10:17:31.573752 (+ 360us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 10:17:31.575748 (+ 1996us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 10:17:31.575749 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 10:17:31.575749 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 10:17:31.576170 (+ 421us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 10:17:31.576385 (+ 215us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 10:17:31.576386 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 10:17:31.576386 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 10:17:31.576449 (+ 63us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 10:17:31.576725 (+ 276us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 10:17:31.576725 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 10:17:31.576761 (+ 36us) client_negotiation.cc:769] Sending connection context
0619 10:17:31.577085 (+ 324us) client_negotiation.cc:240] Negotiation successful
0619 10:17:31.577176 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2867,"thread_start_us":58,"threads_started":1}
Jun 19 10:17:31 dist-test-slave-6pt2 krb5kdc[6853](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864250, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.206.126@KRBTEST.COM for oryx/127.0.206.124@KRBTEST.COM
I20260619 10:17:31.583320 7644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:31.576630 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42046 (local address 127.0.206.124:42647)
0619 10:17:31.576775 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0619 10:17:31.576777 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 10:17:31.576784 (+ 7us) server_negotiation.cc:407] Connection header received
0619 10:17:31.576816 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:17:31.576817 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:17:31.576828 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:17:31.576889 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:17:31.577240 (+ 351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:31.577957 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:31.578586 (+ 629us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:31.578913 (+ 327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:31.578983 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:31.580852 (+ 1869us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 10:17:31.580853 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 10:17:31.580853 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 10:17:31.580858 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0619 10:17:31.582312 (+ 1454us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:17:31.582688 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:17:31.582688 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:17:31.582689 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:17:31.582749 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:17:31.582928 (+ 179us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:17:31.582928 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:17:31.582929 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:17:31.583009 (+ 80us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 10:17:31.583057 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0619 10:17:31.583206 (+ 149us) server_negotiation.cc:299] Negotiation successful
0619 10:17:31.583258 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":123,"thread_start_us":94,"threads_started":1}
I20260619 10:17:31.583381 7456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:31.576559 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.124:42647 (local address 127.0.0.1:42046)
0619 10:17:31.576603 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0619 10:17:31.576613 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 10:17:31.576648 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 10:17:31.576907 (+ 259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 10:17:31.576908 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 10:17:31.577065 (+ 157us) client_negotiation.cc:189] Negotiated authn=SASL
0619 10:17:31.577107 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:31.577109 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:31.578031 (+ 922us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:31.578032 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:31.578505 (+ 473us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:31.578507 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:31.578982 (+ 475us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:31.578982 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:31.579030 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:31.579378 (+ 348us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 10:17:31.579383 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 10:17:31.580779 (+ 1396us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 10:17:31.582379 (+ 1600us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 10:17:31.582380 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 10:17:31.582380 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 10:17:31.582586 (+ 206us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 10:17:31.582811 (+ 225us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 10:17:31.582812 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 10:17:31.582812 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 10:17:31.582874 (+ 62us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 10:17:31.583068 (+ 194us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 10:17:31.583069 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 10:17:31.583111 (+ 42us) client_negotiation.cc:769] Sending connection context
0619 10:17:31.583193 (+ 82us) client_negotiation.cc:240] Negotiation successful
0619 10:17:31.583288 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":30}
I20260619 10:17:31.583820 825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTU9OP6/build/release/bin/kudu with pid 6960
I20260619 10:17:31.584273 6898 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.0.206.124:42647: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "4bac10aeb06347f2bfadf295a3417d44"
member_type: NON_VOTER
last_known_addr {
host: "127.0.206.124"
port: 42647
}
attrs {
promote: true
}
}
dest_uuid: "bd146d3a022940a68274ddac0993c2e3"
cas_config_opid_index: 5
I20260619 10:17:31.584515 6898 consensus_queue.cc:237] T 00000000000000000000000000000000 P bd146d3a022940a68274ddac0993c2e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "bd146d3a022940a68274ddac0993c2e3" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 38663 } } peers { permanent_uuid: "cd27214b90874d35b38271d228518b20" member_type: NON_VOTER last_known_addr { host: "127.0.206.125" port: 40043 } attrs { promote: true } } peers { permanent_uuid: "4bac10aeb06347f2bfadf295a3417d44" member_type: NON_VOTER last_known_addr { host: "127.0.206.124" port: 42647 } attrs { promote: true } }
I20260619 10:17:31.584839 7459 raft_consensus.cc:2932] T 00000000000000000000000000000000 P bd146d3a022940a68274ddac0993c2e3 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 4bac10aeb06347f2bfadf295a3417d44 (127.0.206.124) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "bd146d3a022940a68274ddac0993c2e3" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 38663 } } peers { permanent_uuid: "cd27214b90874d35b38271d228518b20" member_type: NON_VOTER last_known_addr { host: "127.0.206.125" port: 40043 } attrs { promote: true } } peers { permanent_uuid: "4bac10aeb06347f2bfadf295a3417d44" member_type: NON_VOTER last_known_addr { host: "127.0.206.124" port: 42647 } attrs { promote: true } } }
I20260619 10:17:31.585801 7459 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.0.206.124:42647
I20260619 10:17:31.585863 7459 sys_catalog.cc:455] T 00000000000000000000000000000000 P bd146d3a022940a68274ddac0993c2e3 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "bd146d3a022940a68274ddac0993c2e3" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "bd146d3a022940a68274ddac0993c2e3" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 38663 } } peers { permanent_uuid: "cd27214b90874d35b38271d228518b20" member_type: NON_VOTER last_known_addr { host: "127.0.206.125" port: 40043 } attrs { promote: true } } peers { permanent_uuid: "4bac10aeb06347f2bfadf295a3417d44" member_type: NON_VOTER last_known_addr { host: "127.0.206.124" port: 42647 } attrs { promote: true } } }
I20260619 10:17:31.585929 7459 sys_catalog.cc:458] T 00000000000000000000000000000000 P bd146d3a022940a68274ddac0993c2e3 [sys.catalog]: This master's current role is: LEADER
I20260619 10:17:31.586357 7593 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4bac10aeb06347f2bfadf295a3417d44 [term 0 FOLLOWER]: Advancing to term 1
I20260619 10:17:31.587317 7593 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 4bac10aeb06347f2bfadf295a3417d44 [term 1 FOLLOWER]: Refusing update from remote peer bd146d3a022940a68274ddac0993c2e3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260619 10:17:31.587549 7551 master.cc:537] Master@127.0.206.124:42647 shutting down...
I20260619 10:17:31.590548 7458 consensus_queue.cc:1035] T 00000000000000000000000000000000 P bd146d3a022940a68274ddac0993c2e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4bac10aeb06347f2bfadf295a3417d44" member_type: NON_VOTER last_known_addr { host: "127.0.206.124" port: 42647 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 10:17:31.591807 825 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTU9OP6/build/release/bin/kudu
/tmp/dist-test-taskTU9OP6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781864224577058-825-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781864224577058-825-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781864224577058-825-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781864224577058-825-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.206.65:33905
--local_ip_for_outbound_sockets=127.0.206.65
--tserver_master_addrs=127.0.206.126:38663
--webserver_port=39235
--webserver_interface=127.0.206.65
--keytab_file=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781864224577058-825-0/krb5kdc/oryx_127.0.206.65.keytab
--principal=oryx/127.0.206.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.206.84:33273
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.0.206.126:38663,127.0.206.125:40043
--tserver_master_addrs=127.0.206.126:38663,127.0.206.125:40043,127.0.206.124:42647 with env {KRB5CCNAME=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781864224577058-825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781864224577058-825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781864224577058-825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 10:17:31.592257 6884 consensus_peers.cc:487] T 00000000000000000000000000000000 P bd146d3a022940a68274ddac0993c2e3 -> Peer 4bac10aeb06347f2bfadf295a3417d44 (127.0.206.124:42647): Couldn't send request to peer 4bac10aeb06347f2bfadf295a3417d44. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260619 10:17:31.593242 7551 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 4bac10aeb06347f2bfadf295a3417d44 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 10:17:31.593326 7551 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 4bac10aeb06347f2bfadf295a3417d44 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 10:17:31.593371 7551 tablet_replica.cc:330] T 00000000000000000000000000000000 P 4bac10aeb06347f2bfadf295a3417d44: stopping tablet replica
I20260619 10:17:31.595111 7548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:31.587545 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48776 (local address 127.0.206.125:40043)
0619 10:17:31.590303 (+ 2758us) server_negotiation.cc:206] Beginning negotiation
0619 10:17:31.590306 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 10:17:31.590317 (+ 11us) server_negotiation.cc:407] Connection header received
0619 10:17:31.590342 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:17:31.590344 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:17:31.590368 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:17:31.590445 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 10:17:31.590693 (+ 248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:31.591549 (+ 856us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:31.593660 (+ 2111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:31.594256 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:31.594303 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:31.594366 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0619 10:17:31.595009 (+ 643us) server_negotiation.cc:299] Negotiation successful
0619 10:17:31.595046 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2730}
I20260619 10:17:31.595314 7456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:31.586118 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.125:40043 (local address 127.0.0.1:48776)
0619 10:17:31.586161 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0619 10:17:31.586171 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 10:17:31.586208 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 10:17:31.590488 (+ 4280us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 10:17:31.590489 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 10:17:31.590491 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 10:17:31.590529 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:31.590531 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:31.591893 (+ 1362us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:31.591894 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:31.593558 (+ 1664us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:31.593561 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:31.594424 (+ 863us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:31.594425 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:31.594489 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:31.594821 (+ 332us) client_negotiation.cc:769] Sending connection context
0619 10:17:31.594867 (+ 46us) client_negotiation.cc:240] Negotiation successful
0619 10:17:31.595253 (+ 386us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":16}
I20260619 10:17:31.595602 7507 raft_consensus.cc:1261] T 00000000000000000000000000000000 P cd27214b90874d35b38271d228518b20 [term 1 LEARNER]: Refusing update from remote peer bd146d3a022940a68274ddac0993c2e3: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260619 10:17:31.595779 7458 consensus_queue.cc:1035] T 00000000000000000000000000000000 P bd146d3a022940a68274ddac0993c2e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd27214b90874d35b38271d228518b20" member_type: NON_VOTER last_known_addr { host: "127.0.206.125" port: 40043 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260619 10:17:31.596300 7507 raft_consensus.cc:2932] T 00000000000000000000000000000000 P cd27214b90874d35b38271d228518b20 [term 1 LEARNER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 4bac10aeb06347f2bfadf295a3417d44 (127.0.206.124) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "bd146d3a022940a68274ddac0993c2e3" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 38663 } } peers { permanent_uuid: "cd27214b90874d35b38271d228518b20" member_type: NON_VOTER last_known_addr { host: "127.0.206.125" port: 40043 } attrs { promote: true } } peers { permanent_uuid: "4bac10aeb06347f2bfadf295a3417d44" member_type: NON_VOTER last_known_addr { host: "127.0.206.124" port: 42647 } attrs { promote: true } } }
I20260619 10:17:31.597514 7536 sys_catalog.cc:455] T 00000000000000000000000000000000 P cd27214b90874d35b38271d228518b20 [sys.catalog]: SysCatalogTable state changed. Reason: New leader bd146d3a022940a68274ddac0993c2e3. Latest consensus state: current_term: 1 leader_uuid: "bd146d3a022940a68274ddac0993c2e3" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "bd146d3a022940a68274ddac0993c2e3" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 38663 } } peers { permanent_uuid: "cd27214b90874d35b38271d228518b20" member_type: NON_VOTER last_known_addr { host: "127.0.206.125" port: 40043 } attrs { promote: true } } peers { permanent_uuid: "4bac10aeb06347f2bfadf295a3417d44" member_type: NON_VOTER last_known_addr { host: "127.0.206.124" port: 42647 } attrs { promote: true } } }
I20260619 10:17:31.597642 7536 sys_catalog.cc:458] T 00000000000000000000000000000000 P cd27214b90874d35b38271d228518b20 [sys.catalog]: This master's current role is: LEARNER
I20260619 10:17:31.597656 7458 raft_consensus.cc:1050] T 00000000000000000000000000000000 P bd146d3a022940a68274ddac0993c2e3: attempting to promote NON_VOTER cd27214b90874d35b38271d228518b20 to VOTER
I20260619 10:17:31.597683 7536 sys_catalog.cc:455] T 00000000000000000000000000000000 P cd27214b90874d35b38271d228518b20 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "bd146d3a022940a68274ddac0993c2e3" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "bd146d3a022940a68274ddac0993c2e3" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 38663 } } peers { permanent_uuid: "cd27214b90874d35b38271d228518b20" member_type: NON_VOTER last_known_addr { host: "127.0.206.125" port: 40043 } attrs { promote: true } } peers { permanent_uuid: "4bac10aeb06347f2bfadf295a3417d44" member_type: NON_VOTER last_known_addr { host: "127.0.206.124" port: 42647 } attrs { promote: true } } }
I20260619 10:17:31.597740 7536 sys_catalog.cc:458] T 00000000000000000000000000000000 P cd27214b90874d35b38271d228518b20 [sys.catalog]: This master's current role is: LEARNER
I20260619 10:17:31.597832 7458 consensus_queue.cc:237] T 00000000000000000000000000000000 P bd146d3a022940a68274ddac0993c2e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "bd146d3a022940a68274ddac0993c2e3" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 38663 } } peers { permanent_uuid: "cd27214b90874d35b38271d228518b20" member_type: VOTER last_known_addr { host: "127.0.206.125" port: 40043 } attrs { promote: false } } peers { permanent_uuid: "4bac10aeb06347f2bfadf295a3417d44" member_type: NON_VOTER last_known_addr { host: "127.0.206.124" port: 42647 } attrs { promote: true } }
I20260619 10:17:31.598286 7456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:31.598151 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.124:42647 (local address 127.0.0.1:42060)
0619 10:17:31.598204 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0619 10:17:31.598252 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.124:42647: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260619 10:17:31.598287 7507 raft_consensus.cc:1261] T 00000000000000000000000000000000 P cd27214b90874d35b38271d228518b20 [term 1 LEARNER]: Refusing update from remote peer bd146d3a022940a68274ddac0993c2e3: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260619 10:17:31.598557 7457 consensus_queue.cc:1035] T 00000000000000000000000000000000 P bd146d3a022940a68274ddac0993c2e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd27214b90874d35b38271d228518b20" member_type: VOTER last_known_addr { host: "127.0.206.125" port: 40043 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260619 10:17:31.598924 7456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:31.598683 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.124:42647 (local address 127.0.0.1:42076)
0619 10:17:31.598864 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0619 10:17:31.598893 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.124:42647: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":166}
W20260619 10:17:31.599050 6884 consensus_peers.cc:487] T 00000000000000000000000000000000 P bd146d3a022940a68274ddac0993c2e3 -> Peer 4bac10aeb06347f2bfadf295a3417d44 (127.0.206.124:42647): Couldn't send request to peer 4bac10aeb06347f2bfadf295a3417d44. Status: Network error: Client connection negotiation failed: client connection to 127.0.206.124:42647: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260619 10:17:31.599243 7459 raft_consensus.cc:2932] T 00000000000000000000000000000000 P bd146d3a022940a68274ddac0993c2e3 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, cd27214b90874d35b38271d228518b20 (127.0.206.125) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "bd146d3a022940a68274ddac0993c2e3" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 38663 } } peers { permanent_uuid: "cd27214b90874d35b38271d228518b20" member_type: VOTER last_known_addr { host: "127.0.206.125" port: 40043 } attrs { promote: false } } peers { permanent_uuid: "4bac10aeb06347f2bfadf295a3417d44" member_type: NON_VOTER last_known_addr { host: "127.0.206.124" port: 42647 } attrs { promote: true } } }
I20260619 10:17:31.600049 7459 sys_catalog.cc:455] T 00000000000000000000000000000000 P bd146d3a022940a68274ddac0993c2e3 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "bd146d3a022940a68274ddac0993c2e3" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "bd146d3a022940a68274ddac0993c2e3" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 38663 } } peers { permanent_uuid: "cd27214b90874d35b38271d228518b20" member_type: VOTER last_known_addr { host: "127.0.206.125" port: 40043 } attrs { promote: false } } peers { permanent_uuid: "4bac10aeb06347f2bfadf295a3417d44" member_type: NON_VOTER last_known_addr { host: "127.0.206.124" port: 42647 } attrs { promote: true } } }
I20260619 10:17:31.600131 7459 sys_catalog.cc:458] T 00000000000000000000000000000000 P bd146d3a022940a68274ddac0993c2e3 [sys.catalog]: This master's current role is: LEADER
I20260619 10:17:31.600183 7459 sys_catalog.cc:455] T 00000000000000000000000000000000 P bd146d3a022940a68274ddac0993c2e3 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "bd146d3a022940a68274ddac0993c2e3" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "bd146d3a022940a68274ddac0993c2e3" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 38663 } } peers { permanent_uuid: "cd27214b90874d35b38271d228518b20" member_type: VOTER last_known_addr { host: "127.0.206.125" port: 40043 } attrs { promote: false } } peers { permanent_uuid: "4bac10aeb06347f2bfadf295a3417d44" member_type: NON_VOTER last_known_addr { host: "127.0.206.124" port: 42647 } attrs { promote: true } } }
I20260619 10:17:31.600230 7459 sys_catalog.cc:458] T 00000000000000000000000000000000 P bd146d3a022940a68274ddac0993c2e3 [sys.catalog]: This master's current role is: LEADER
I20260619 10:17:31.600479 7507 raft_consensus.cc:2932] T 00000000000000000000000000000000 P cd27214b90874d35b38271d228518b20 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, cd27214b90874d35b38271d228518b20 (127.0.206.125) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "bd146d3a022940a68274ddac0993c2e3" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 38663 } } peers { permanent_uuid: "cd27214b90874d35b38271d228518b20" member_type: VOTER last_known_addr { host: "127.0.206.125" port: 40043 } attrs { promote: false } } peers { permanent_uuid: "4bac10aeb06347f2bfadf295a3417d44" member_type: NON_VOTER last_known_addr { host: "127.0.206.124" port: 42647 } attrs { promote: true } } }
I20260619 10:17:31.601380 7647 sys_catalog.cc:455] T 00000000000000000000000000000000 P cd27214b90874d35b38271d228518b20 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "bd146d3a022940a68274ddac0993c2e3" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "bd146d3a022940a68274ddac0993c2e3" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 38663 } } peers { permanent_uuid: "cd27214b90874d35b38271d228518b20" member_type: VOTER last_known_addr { host: "127.0.206.125" port: 40043 } attrs { promote: false } } peers { permanent_uuid: "4bac10aeb06347f2bfadf295a3417d44" member_type: NON_VOTER last_known_addr { host: "127.0.206.124" port: 42647 } attrs { promote: true } } }
I20260619 10:17:31.601464 7647 sys_catalog.cc:458] T 00000000000000000000000000000000 P cd27214b90874d35b38271d228518b20 [sys.catalog]: This master's current role is: FOLLOWER
I20260619 10:17:31.606528 7551 master.cc:552] Master@127.0.206.124:42647 shutdown complete.
I20260619 10:17:31.606575 7551 master_runner.cc:305] Clearing existing system tablet
I20260619 10:17:31.606729 7551 fs_manager.cc:359] Metadata directory not provided
I20260619 10:17:31.606775 7551 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 10:17:31.607491 7551 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.001s sys 0.000s
I20260619 10:17:31.608008 7649 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 10:17:31.608171 7551 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781864224577058-825-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 10:17:31.608228 7551 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260619 10:17:31.608251 7551 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781864224577058-825-0/minicluster-data/master-2/data,/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781864224577058-825-0/minicluster-data/master-2/wal
uuid: "4bac10aeb06347f2bfadf295a3417d44"
format_stamp: "Formatted at 2026-06-19 10:17:31 on dist-test-slave-6pt2"
server_key: "63d6bacf0174d66832d11af48c1bc1f4"
server_key_iv: "bc3333f62bdbbc90389129e6ff596439"
server_key_version: "encryptionkey@0"
I20260619 10:17:31.609040 7551 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 4bac10aeb06347f2bfadf295a3417d44: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 10:17:31.610253 7551 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 4bac10aeb06347f2bfadf295a3417d44: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260619 10:17:31.610317 7551 log.cc:1196] T 00000000000000000000000000000000 P 4bac10aeb06347f2bfadf295a3417d44: Deleting WAL directory at /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781864224577058-825-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260619 10:17:31.610496 7551 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 4bac10aeb06347f2bfadf295a3417d44: Deleting consensus metadata
I20260619 10:17:31.610687 7551 master_runner.cc:315] Copying system tablet from 127.0.206.126:38663
I20260619 10:17:31.611239 7551 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 4bac10aeb06347f2bfadf295a3417d44: tablet copy: Beginning tablet copy session from remote peer at address 127.0.206.126:38663
I20260619 10:17:31.618120 6956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:31.611533 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37434 (local address 127.0.206.126:38663)
0619 10:17:31.611587 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0619 10:17:31.611589 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 10:17:31.611677 (+ 88us) server_negotiation.cc:407] Connection header received
0619 10:17:31.611726 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:17:31.611727 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:17:31.611736 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:17:31.611797 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:17:31.612117 (+ 320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:31.612884 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:31.613907 (+ 1023us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:31.614213 (+ 306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:31.614283 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:31.615337 (+ 1054us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 10:17:31.615338 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 10:17:31.615339 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 10:17:31.615346 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 10:17:31.616958 (+ 1612us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:17:31.617361 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:17:31.617362 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:17:31.617363 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:17:31.617420 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:17:31.617632 (+ 212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:17:31.617633 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:17:31.617633 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:17:31.617746 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 10:17:31.617785 (+ 39us) server_negotiation.cc:1035] Waiting for connection context
0619 10:17:31.617970 (+ 185us) server_negotiation.cc:299] Negotiation successful
0619 10:17:31.618042 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260619 10:17:31.618403 7655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:31.611472 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:38663 (local address 127.0.0.1:37434)
0619 10:17:31.611610 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0619 10:17:31.611621 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 10:17:31.611680 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 10:17:31.611799 (+ 119us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 10:17:31.611800 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 10:17:31.611987 (+ 187us) client_negotiation.cc:189] Negotiated authn=SASL
0619 10:17:31.612041 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:31.612042 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:31.612968 (+ 926us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:31.612969 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:31.613822 (+ 853us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:31.613825 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:31.614299 (+ 474us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:31.614300 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:31.614369 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:31.614857 (+ 488us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 10:17:31.614862 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 10:17:31.615247 (+ 385us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 10:17:31.617046 (+ 1799us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 10:17:31.617047 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 10:17:31.617047 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 10:17:31.617278 (+ 231us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 10:17:31.617494 (+ 216us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 10:17:31.617495 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 10:17:31.617496 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 10:17:31.617563 (+ 67us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 10:17:31.617829 (+ 266us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 10:17:31.617829 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 10:17:31.617871 (+ 42us) client_negotiation.cc:769] Sending connection context
0619 10:17:31.618227 (+ 356us) client_negotiation.cc:240] Negotiation successful
0619 10:17:31.618321 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":82,"threads_started":1}
I20260619 10:17:31.621179 6918 tablet_copy_service.cc:140] P bd146d3a022940a68274ddac0993c2e3: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 4bac10aeb06347f2bfadf295a3417d44 ({username='oryx', principal='oryx/127.0.206.124@KRBTEST.COM'} at 127.0.0.1:37434)
I20260619 10:17:31.621240 6918 tablet_copy_service.cc:161] P bd146d3a022940a68274ddac0993c2e3: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 4bac10aeb06347f2bfadf295a3417d44 at {username='oryx', principal='oryx/127.0.206.124@KRBTEST.COM'} at 127.0.0.1:37434: session id = 4bac10aeb06347f2bfadf295a3417d44-00000000000000000000000000000000
I20260619 10:17:31.621753 6918 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P bd146d3a022940a68274ddac0993c2e3: Tablet Copy: opened 0 blocks and 1 log segments
I20260619 10:17:31.622205 7551 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260619 10:17:31.623411 7551 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 4bac10aeb06347f2bfadf295a3417d44: tablet copy: Starting download of 0 data blocks...
I20260619 10:17:31.623553 7551 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 4bac10aeb06347f2bfadf295a3417d44: tablet copy: Starting download of 1 WAL segments...
I20260619 10:17:31.624234 7551 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 4bac10aeb06347f2bfadf295a3417d44: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260619 10:17:31.625048 6918 tablet_copy_service.cc:342] P bd146d3a022940a68274ddac0993c2e3: Request end of tablet copy session 4bac10aeb06347f2bfadf295a3417d44-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.0.206.124@KRBTEST.COM'} at 127.0.0.1:37434
I20260619 10:17:31.625097 6918 tablet_copy_service.cc:433] P bd146d3a022940a68274ddac0993c2e3: ending tablet copy session 4bac10aeb06347f2bfadf295a3417d44-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 4bac10aeb06347f2bfadf295a3417d44
I20260619 10:17:31.630353 7551 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 10:17:31.631594 7656 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 10:17:31.631629 7657 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 10:17:31.635452 7659 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 10:17:31.639416 7551 server_base.cc:1034] running on GCE node
I20260619 10:17:31.639537 7551 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 10:17:31.639678 7551 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 10:17:31.640821 7551 hybrid_clock.cc:648] HybridClock initialized: now 1781864251640829 us; error 51 us; skew 500 ppm
Jun 19 10:17:31 dist-test-slave-6pt2 krb5kdc[6853](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864251, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.206.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 10:17:31.642273 7551 init.cc:377] Logged in from keytab as oryx/127.0.206.124@KRBTEST.COM (short username oryx)
I20260619 10:17:31.642517 7551 webserver.cc:460] Webserver started at http://127.0.206.124:36655/ using document root <none> and password file <none>
I20260619 10:17:31.642709 7551 fs_manager.cc:359] Metadata directory not provided
I20260619 10:17:31.642740 7551 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 10:17:31.643443 7551 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.000s sys 0.001s
I20260619 10:17:31.644042 7666 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 10:17:31.644176 7551 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260619 10:17:31.644235 7551 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781864224577058-825-0/minicluster-data/master-2/data,/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781864224577058-825-0/minicluster-data/master-2/wal
uuid: "4bac10aeb06347f2bfadf295a3417d44"
format_stamp: "Formatted at 2026-06-19 10:17:31 on dist-test-slave-6pt2"
server_key: "63d6bacf0174d66832d11af48c1bc1f4"
server_key_iv: "bc3333f62bdbbc90389129e6ff596439"
server_key_version: "encryptionkey@0"
I20260619 10:17:31.644296 7551 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781864224577058-825-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781864224577058-825-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781864224577058-825-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 10:17:31.651244 7551 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 10:17:31.652945 7551 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 10:17:31.655936 7551 rpc_server.cc:307] RPC server started. Bound to: 127.0.206.124:42647
I20260619 10:17:31.656317 7551 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781864224577058-825-0/minicluster-data/master-2/data/info.pb
I20260619 10:17:31.657126 7719 sys_catalog.cc:263] Verifying existing consensus state
I20260619 10:17:31.657589 7719 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4bac10aeb06347f2bfadf295a3417d44: Bootstrap starting.
I20260619 10:17:31.658723 7718 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.206.124:42647 every 8 connection(s)
I20260619 10:17:31.660399 7719 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4bac10aeb06347f2bfadf295a3417d44: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260619 10:17:31.660615 7719 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4bac10aeb06347f2bfadf295a3417d44: Bootstrap complete.
I20260619 10:17:31.660818 7719 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4bac10aeb06347f2bfadf295a3417d44 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "bd146d3a022940a68274ddac0993c2e3" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 38663 } } peers { permanent_uuid: "cd27214b90874d35b38271d228518b20" member_type: VOTER last_known_addr { host: "127.0.206.125" port: 40043 } attrs { promote: false } } peers { permanent_uuid: "4bac10aeb06347f2bfadf295a3417d44" member_type: NON_VOTER last_known_addr { host: "127.0.206.124" port: 42647 } attrs { promote: true } }
I20260619 10:17:31.660907 7719 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4bac10aeb06347f2bfadf295a3417d44 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 4bac10aeb06347f2bfadf295a3417d44, State: Initialized, Role: LEARNER
I20260619 10:17:31.660943 7719 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4bac10aeb06347f2bfadf295a3417d44 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "bd146d3a022940a68274ddac0993c2e3" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 38663 } } peers { permanent_uuid: "cd27214b90874d35b38271d228518b20" member_type: VOTER last_known_addr { host: "127.0.206.125" port: 40043 } attrs { promote: false } } peers { permanent_uuid: "4bac10aeb06347f2bfadf295a3417d44" member_type: NON_VOTER last_known_addr { host: "127.0.206.124" port: 42647 } attrs { promote: true } }
I20260619 10:17:31.661111 7719 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4bac10aeb06347f2bfadf295a3417d44 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 10:17:31.661213 7722 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4bac10aeb06347f2bfadf295a3417d44 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "bd146d3a022940a68274ddac0993c2e3" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 38663 } } peers { permanent_uuid: "cd27214b90874d35b38271d228518b20" member_type: VOTER last_known_addr { host: "127.0.206.125" port: 40043 } attrs { promote: false } } peers { permanent_uuid: "4bac10aeb06347f2bfadf295a3417d44" member_type: NON_VOTER last_known_addr { host: "127.0.206.124" port: 42647 } attrs { promote: true } } }
I20260619 10:17:31.661353 7722 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4bac10aeb06347f2bfadf295a3417d44 [sys.catalog]: This master's current role is: LEARNER
I20260619 10:17:31.662523 7733 catalog_manager.cc:1259] Loaded cluster ID: a825fc644b564a9897f7dc902ab7b888
I20260619 10:17:31.663210 7733 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 4bac10aeb06347f2bfadf295a3417d44: loading cluster ID for follower catalog manager: success
I20260619 10:17:31.664251 7733 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 4bac10aeb06347f2bfadf295a3417d44: acquiring CA information for follower catalog manager: success
I20260619 10:17:31.664521 7733 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 4bac10aeb06347f2bfadf295a3417d44: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260619 10:17:31.688000 7645 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 10:17:31.688201 7645 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 10:17:31.688223 7645 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 10:17:31.689412 7645 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 10:17:31.689450 7645 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 10:17:31.689476 7645 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.206.65
I20260619 10:17:31.690955 7645 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.206.84:33273
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781864224577058-825-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781864224577058-825-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781864224577058-825-0/krb5kdc/oryx_127.0.206.65.keytab
--principal=oryx/127.0.206.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.206.65:33905
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781864224577058-825-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.206.65
--webserver_port=39235
--tserver_master_addrs=127.0.206.126:38663,127.0.206.125:40043,127.0.206.124:42647
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7645
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.206.65
--log_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781864224577058-825-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 09:32:44 UTC on ccycloud.quasar-gbtldp.root.comops.site
build id 4455440
I20260619 10:17:31.691183 7645 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 10:17:31.691408 7645 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 10:17:31.696259 7739 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 10:17:31.696281 7738 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 10:17:31.696451 7645 server_base.cc:1034] running on GCE node
W20260619 10:17:31.696278 7741 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 10:17:31.696730 7645 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 10:17:31.696955 7645 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 10:17:31.698112 7645 hybrid_clock.cc:648] HybridClock initialized: now 1781864251698108 us; error 36 us; skew 500 ppm
Jun 19 10:17:31 dist-test-slave-6pt2 krb5kdc[6853](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864251, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.206.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 10:17:31.700523 7645 init.cc:377] Logged in from keytab as oryx/127.0.206.65@KRBTEST.COM (short username oryx)
I20260619 10:17:31.700945 7645 webserver.cc:460] Webserver started at http://127.0.206.65:39235/ using document root <none> and password file <none>
I20260619 10:17:31.701133 7645 fs_manager.cc:359] Metadata directory not provided
I20260619 10:17:31.701184 7645 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 10:17:31.702369 7645 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260619 10:17:31.702921 7748 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 10:17:31.703071 7645 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260619 10:17:31.703150 7645 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781864224577058-825-0/minicluster-data/ts-0/data,/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781864224577058-825-0/minicluster-data/ts-0/wal
uuid: "67adfa875ae04601b7b8fceb7610166c"
format_stamp: "Formatted at 2026-06-19 10:17:30 on dist-test-slave-6pt2"
server_key: "e299581acb30e48f030e0da0d25b4d44"
server_key_iv: "8f80a3ec3d65fcefdf0d57ab42ec4c02"
server_key_version: "encryptionkey@0"
I20260619 10:17:31.703440 7645 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781864224577058-825-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781864224577058-825-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781864224577058-825-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 10:17:31.716820 7645 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 10:17:31.718407 7645 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 10:17:31.718509 7645 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 10:17:31.718736 7645 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 10:17:31.718966 7645 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 10:17:31.718997 7645 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 10:17:31.719017 7645 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 10:17:31.719029 7645 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 10:17:31.724776 7645 rpc_server.cc:307] RPC server started. Bound to: 127.0.206.65:33905
I20260619 10:17:31.724838 7861 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.206.65:33905 every 8 connection(s)
I20260619 10:17:31.725179 7645 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781864224577058-825-0/minicluster-data/ts-0/data/info.pb
I20260619 10:17:31.729827 825 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTU9OP6/build/release/bin/kudu as pid 7645
I20260619 10:17:31.729970 825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTU9OP6/build/release/bin/kudu with pid 7094
Jun 19 10:17:31 dist-test-slave-6pt2 krb5kdc[6853](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864251, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.206.65@KRBTEST.COM for oryx/127.0.206.126@KRBTEST.COM
Jun 19 10:17:31 dist-test-slave-6pt2 krb5kdc[6853](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864251, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.206.65@KRBTEST.COM for oryx/127.0.206.124@KRBTEST.COM
Jun 19 10:17:31 dist-test-slave-6pt2 krb5kdc[6853](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864251, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.206.65@KRBTEST.COM for oryx/127.0.206.125@KRBTEST.COM
I20260619 10:17:31.736588 825 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTU9OP6/build/release/bin/kudu
/tmp/dist-test-taskTU9OP6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781864224577058-825-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781864224577058-825-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781864224577058-825-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781864224577058-825-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.206.66:37571
--local_ip_for_outbound_sockets=127.0.206.66
--tserver_master_addrs=127.0.206.126:38663
--webserver_port=38305
--webserver_interface=127.0.206.66
--keytab_file=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781864224577058-825-0/krb5kdc/oryx_127.0.206.66.keytab
--principal=oryx/127.0.206.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.206.84:33273
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.0.206.126:38663,127.0.206.125:40043
--tserver_master_addrs=127.0.206.126:38663,127.0.206.125:40043,127.0.206.124:42647 with env {KRB5CCNAME=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781864224577058-825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781864224577058-825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781864224577058-825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 10:17:31.736971 6956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:31.726349 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.65:59943 (local address 127.0.206.126:38663)
0619 10:17:31.726415 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0619 10:17:31.726417 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 10:17:31.726459 (+ 42us) server_negotiation.cc:407] Connection header received
0619 10:17:31.726900 (+ 441us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:17:31.726901 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:17:31.726913 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:17:31.726949 (+ 36us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:17:31.727651 (+ 702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:31.728193 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:31.729640 (+ 1447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:31.729933 (+ 293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:31.729977 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:31.733117 (+ 3140us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 10:17:31.733119 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 10:17:31.733119 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 10:17:31.733127 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 10:17:31.735307 (+ 2180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:17:31.735821 (+ 514us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:17:31.735822 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:17:31.735822 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:17:31.735894 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:17:31.736181 (+ 287us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:17:31.736182 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:17:31.736182 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:17:31.736334 (+ 152us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 10:17:31.736377 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0619 10:17:31.736748 (+ 371us) server_negotiation.cc:299] Negotiation successful
0619 10:17:31.736831 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260619 10:17:31.737159 7869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:31.726283 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:38663 (local address 127.0.206.65:59943)
0619 10:17:31.726389 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0619 10:17:31.726400 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 10:17:31.726786 (+ 386us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 10:17:31.727022 (+ 236us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 10:17:31.727023 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 10:17:31.727456 (+ 433us) client_negotiation.cc:189] Negotiated authn=SASL
0619 10:17:31.727532 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:31.727538 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:31.728296 (+ 758us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:31.728297 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:31.729538 (+ 1241us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:31.729541 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:31.730116 (+ 575us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:31.730116 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:31.730198 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:31.730932 (+ 734us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 10:17:31.730942 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0619 10:17:31.733018 (+ 2076us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 10:17:31.735435 (+ 2417us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 10:17:31.735437 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 10:17:31.735437 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 10:17:31.735731 (+ 294us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 10:17:31.735961 (+ 230us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 10:17:31.735962 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 10:17:31.735962 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 10:17:31.736094 (+ 132us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 10:17:31.736424 (+ 330us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 10:17:31.736429 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 10:17:31.736577 (+ 148us) client_negotiation.cc:769] Sending connection context
0619 10:17:31.736712 (+ 135us) client_negotiation.cc:240] Negotiation successful
0619 10:17:31.736890 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":92,"mutex_wait_us":6,"thread_start_us":60,"threads_started":1}
I20260619 10:17:31.737566 7863 heartbeater.cc:347] Connected to a master server at 127.0.206.126:38663
I20260619 10:17:31.737653 7863 heartbeater.cc:464] Registering TS with master...
I20260619 10:17:31.737728 7866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:31.725843 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.65:43223 (local address 127.0.206.124:42647)
0619 10:17:31.725959 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0619 10:17:31.725961 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 10:17:31.726105 (+ 144us) server_negotiation.cc:407] Connection header received
0619 10:17:31.726900 (+ 795us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:17:31.726901 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:17:31.726914 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:17:31.726956 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:17:31.727639 (+ 683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:31.728439 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:31.729639 (+ 1200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:31.729957 (+ 318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:31.729993 (+ 36us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:31.733459 (+ 3466us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 10:17:31.733460 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 10:17:31.733461 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 10:17:31.733468 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 10:17:31.735219 (+ 1751us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:17:31.735821 (+ 602us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:17:31.735822 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:17:31.735822 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:17:31.736141 (+ 319us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:17:31.736340 (+ 199us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:17:31.736341 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:17:31.736341 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:17:31.736465 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 10:17:31.736965 (+ 500us) server_negotiation.cc:1035] Waiting for connection context
0619 10:17:31.737552 (+ 587us) server_negotiation.cc:299] Negotiation successful
0619 10:17:31.737635 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":91,"thread_start_us":60,"threads_started":1}
I20260619 10:17:31.737818 7863 heartbeater.cc:511] Master 127.0.206.126:38663 requested a full tablet report, sending...
I20260619 10:17:31.738205 7867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:31.725855 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.124:42647 (local address 127.0.206.65:43223)
0619 10:17:31.725994 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0619 10:17:31.726030 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0619 10:17:31.726786 (+ 756us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 10:17:31.727020 (+ 234us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 10:17:31.727021 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 10:17:31.727351 (+ 330us) client_negotiation.cc:189] Negotiated authn=SASL
0619 10:17:31.727532 (+ 181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:31.727544 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:31.728526 (+ 982us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:31.728527 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:31.729545 (+ 1018us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:31.729546 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:31.730067 (+ 521us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:31.730068 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:31.730198 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:31.731042 (+ 844us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 10:17:31.731047 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 10:17:31.733348 (+ 2301us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 10:17:31.735313 (+ 1965us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 10:17:31.735319 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 10:17:31.735321 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 10:17:31.735731 (+ 410us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 10:17:31.736219 (+ 488us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 10:17:31.736220 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 10:17:31.736220 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 10:17:31.736270 (+ 50us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 10:17:31.737374 (+ 1104us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 10:17:31.737375 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 10:17:31.737417 (+ 42us) client_negotiation.cc:769] Sending connection context
0619 10:17:31.738018 (+ 601us) client_negotiation.cc:240] Negotiation successful
0619 10:17:31.738122 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":103,"mutex_wait_us":21,"thread_start_us":79,"threads_started":1}
I20260619 10:17:31.738205 6898 ts_manager.cc:194] Re-registered known tserver with Master: 67adfa875ae04601b7b8fceb7610166c (127.0.206.65:33905)
I20260619 10:17:31.738688 6898 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.0.206.65@KRBTEST.COM'} at 127.0.206.65:59943
I20260619 10:17:31.738778 7862 heartbeater.cc:347] Connected to a master server at 127.0.206.124:42647
I20260619 10:17:31.738826 7862 heartbeater.cc:464] Registering TS with master...
I20260619 10:17:31.738888 7862 heartbeater.cc:511] Master 127.0.206.124:42647 requested a full tablet report, sending...
I20260619 10:17:31.739059 7548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:31.726195 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.65:35595 (local address 127.0.206.125:40043)
0619 10:17:31.726241 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0619 10:17:31.726243 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 10:17:31.726333 (+ 90us) server_negotiation.cc:407] Connection header received
0619 10:17:31.726876 (+ 543us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:17:31.726877 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:17:31.726888 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:17:31.726943 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:17:31.728279 (+ 1336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:31.728824 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:31.729633 (+ 809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:31.729928 (+ 295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:31.729993 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:31.735668 (+ 5675us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 10:17:31.735669 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 10:17:31.735670 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 10:17:31.735677 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 10:17:31.737779 (+ 2102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:17:31.738160 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:17:31.738162 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:17:31.738162 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:17:31.738217 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:17:31.738408 (+ 191us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:17:31.738409 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:17:31.738409 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:17:31.738555 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 10:17:31.738648 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0619 10:17:31.738881 (+ 233us) server_negotiation.cc:299] Negotiation successful
0619 10:17:31.738975 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260619 10:17:31.739271 7683 ts_manager.cc:194] Registered new tserver with Master: 67adfa875ae04601b7b8fceb7610166c (127.0.206.65:33905)
I20260619 10:17:31.739391 7868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:31.726129 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.125:40043 (local address 127.0.206.65:35595)
0619 10:17:31.726230 (+ 101us) negotiation.cc:107] Waiting for socket to connect
0619 10:17:31.726273 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0619 10:17:31.726785 (+ 512us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 10:17:31.727011 (+ 226us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 10:17:31.727015 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 10:17:31.727337 (+ 322us) client_negotiation.cc:189] Negotiated authn=SASL
0619 10:17:31.727532 (+ 195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:31.727538 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:31.728887 (+ 1349us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:31.728888 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:31.729537 (+ 649us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:31.729539 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:31.730052 (+ 513us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:31.730053 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:31.730198 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:31.733946 (+ 3748us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 10:17:31.733951 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 10:17:31.735571 (+ 1620us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 10:17:31.737879 (+ 2308us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 10:17:31.737880 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 10:17:31.737881 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 10:17:31.738066 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 10:17:31.738286 (+ 220us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 10:17:31.738287 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 10:17:31.738287 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 10:17:31.738342 (+ 55us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 10:17:31.738630 (+ 288us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 10:17:31.738630 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 10:17:31.738745 (+ 115us) client_negotiation.cc:769] Sending connection context
0619 10:17:31.739169 (+ 424us) client_negotiation.cc:240] Negotiation successful
0619 10:17:31.739301 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":85,"mutex_wait_us":43,"thread_start_us":64,"threads_started":1}
I20260619 10:17:31.739719 7864 heartbeater.cc:347] Connected to a master server at 127.0.206.125:40043
I20260619 10:17:31.739768 7864 heartbeater.cc:464] Registering TS with master...
I20260619 10:17:31.739835 7864 heartbeater.cc:511] Master 127.0.206.125:40043 requested a full tablet report, sending...
I20260619 10:17:31.740182 7497 ts_manager.cc:194] Registered new tserver with Master: 67adfa875ae04601b7b8fceb7610166c (127.0.206.65:33905)
W20260619 10:17:31.817051 7870 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 10:17:31.817233 7870 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 10:17:31.817255 7870 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 10:17:31.818305 7870 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 10:17:31.818339 7870 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 10:17:31.818364 7870 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.206.66
I20260619 10:17:31.819758 7870 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.206.84:33273
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781864224577058-825-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781864224577058-825-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781864224577058-825-0/krb5kdc/oryx_127.0.206.66.keytab
--principal=oryx/127.0.206.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.206.66:37571
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781864224577058-825-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.206.66
--webserver_port=38305
--tserver_master_addrs=127.0.206.126:38663,127.0.206.125:40043,127.0.206.124:42647
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7870
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.206.66
--log_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781864224577058-825-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 09:32:44 UTC on ccycloud.quasar-gbtldp.root.comops.site
build id 4455440
I20260619 10:17:31.819926 7870 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 10:17:31.820096 7870 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 10:17:31.824704 7875 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 10:17:31.824785 7878 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 10:17:31.824785 7876 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 10:17:31.824858 7870 server_base.cc:1034] running on GCE node
I20260619 10:17:31.825127 7870 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 10:17:31.825359 7870 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 10:17:31.826535 7870 hybrid_clock.cc:648] HybridClock initialized: now 1781864251826513 us; error 44 us; skew 500 ppm
Jun 19 10:17:31 dist-test-slave-6pt2 krb5kdc[6853](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864251, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.206.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 10:17:31.829093 7870 init.cc:377] Logged in from keytab as oryx/127.0.206.66@KRBTEST.COM (short username oryx)
I20260619 10:17:31.829536 7870 webserver.cc:460] Webserver started at http://127.0.206.66:38305/ using document root <none> and password file <none>
I20260619 10:17:31.829752 7870 fs_manager.cc:359] Metadata directory not provided
I20260619 10:17:31.829823 7870 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 10:17:31.831122 7870 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260619 10:17:31.831655 7885 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 10:17:31.831833 7870 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260619 10:17:31.831918 7870 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781864224577058-825-0/minicluster-data/ts-1/data,/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781864224577058-825-0/minicluster-data/ts-1/wal
uuid: "04a062435b8049fa92d4f33097d4e8a2"
format_stamp: "Formatted at 2026-06-19 10:17:30 on dist-test-slave-6pt2"
server_key: "bdf4ff498a51788d1fe38f0489c5e00a"
server_key_iv: "70df18662e03f5d78aa0770ca5d89f9e"
server_key_version: "encryptionkey@0"
I20260619 10:17:31.832207 7870 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781864224577058-825-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781864224577058-825-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781864224577058-825-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 10:17:31.838325 7870 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 10:17:31.839998 7870 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 10:17:31.840138 7870 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 10:17:31.840337 7870 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 10:17:31.840607 7870 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 10:17:31.840638 7870 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 10:17:31.840659 7870 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 10:17:31.840699 7870 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 10:17:31.847390 7870 rpc_server.cc:307] RPC server started. Bound to: 127.0.206.66:37571
I20260619 10:17:31.847426 7998 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.206.66:37571 every 8 connection(s)
I20260619 10:17:31.847754 7870 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781864224577058-825-0/minicluster-data/ts-1/data/info.pb
I20260619 10:17:31.852865 825 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTU9OP6/build/release/bin/kudu as pid 7870
I20260619 10:17:31.853042 825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTU9OP6/build/release/bin/kudu with pid 7228
Jun 19 10:17:31 dist-test-slave-6pt2 krb5kdc[6853](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864251, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.206.66@KRBTEST.COM for oryx/127.0.206.125@KRBTEST.COM
Jun 19 10:17:31 dist-test-slave-6pt2 krb5kdc[6853](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864251, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.206.66@KRBTEST.COM for oryx/127.0.206.126@KRBTEST.COM
Jun 19 10:17:31 dist-test-slave-6pt2 krb5kdc[6853](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864251, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.206.66@KRBTEST.COM for oryx/127.0.206.124@KRBTEST.COM
I20260619 10:17:31.858569 7548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:31.848555 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.66:50647 (local address 127.0.206.125:40043)
0619 10:17:31.848597 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0619 10:17:31.848599 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 10:17:31.848858 (+ 259us) server_negotiation.cc:407] Connection header received
0619 10:17:31.849602 (+ 744us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:17:31.849603 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:17:31.849616 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:17:31.849650 (+ 34us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:17:31.850332 (+ 682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:31.851100 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:31.852197 (+ 1097us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:31.852472 (+ 275us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:31.852545 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:31.854984 (+ 2439us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 10:17:31.854986 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 10:17:31.854986 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 10:17:31.854993 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 10:17:31.856802 (+ 1809us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:17:31.857506 (+ 704us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:17:31.857507 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:17:31.857507 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:17:31.857564 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:17:31.857817 (+ 253us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:17:31.857817 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:17:31.857818 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:17:31.857950 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 10:17:31.857993 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0619 10:17:31.858340 (+ 347us) server_negotiation.cc:299] Negotiation successful
0619 10:17:31.858432 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260619 10:17:31.859143 8003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:31.848576 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.125:40043 (local address 127.0.206.66:50647)
0619 10:17:31.848731 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0619 10:17:31.848764 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0619 10:17:31.849483 (+ 719us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 10:17:31.849722 (+ 239us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 10:17:31.849725 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 10:17:31.850055 (+ 330us) client_negotiation.cc:189] Negotiated authn=SASL
0619 10:17:31.850226 (+ 171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:31.850232 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:31.851190 (+ 958us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:31.851190 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:31.852117 (+ 927us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:31.852118 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:31.852556 (+ 438us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:31.852557 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:31.852619 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:31.853038 (+ 419us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 10:17:31.853043 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 10:17:31.854874 (+ 1831us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 10:17:31.856890 (+ 2016us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 10:17:31.856896 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 10:17:31.856898 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 10:17:31.857206 (+ 308us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 10:17:31.857630 (+ 424us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 10:17:31.857630 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 10:17:31.857630 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 10:17:31.857752 (+ 122us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 10:17:31.858036 (+ 284us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 10:17:31.858041 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 10:17:31.858197 (+ 156us) client_negotiation.cc:769] Sending connection context
0619 10:17:31.858719 (+ 522us) client_negotiation.cc:240] Negotiation successful
0619 10:17:31.858891 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"mutex_wait_us":58,"thread_start_us":80,"threads_started":1}
I20260619 10:17:31.859236 6956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:31.848914 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.66:45775 (local address 127.0.206.126:38663)
0619 10:17:31.848961 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0619 10:17:31.848963 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 10:17:31.849087 (+ 124us) server_negotiation.cc:407] Connection header received
0619 10:17:31.849590 (+ 503us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:17:31.849591 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:17:31.849604 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:17:31.849655 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:17:31.850325 (+ 670us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:31.850872 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:31.851787 (+ 915us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:31.851974 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:31.852045 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:31.856233 (+ 4188us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 10:17:31.856235 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 10:17:31.856235 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 10:17:31.856242 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 10:17:31.857729 (+ 1487us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:17:31.858129 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:17:31.858130 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:17:31.858131 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:17:31.858186 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:17:31.858400 (+ 214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:17:31.858401 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:17:31.858401 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:17:31.858537 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 10:17:31.858580 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0619 10:17:31.859044 (+ 464us) server_negotiation.cc:299] Negotiation successful
0619 10:17:31.859122 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260619 10:17:31.859901 8004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:31.848805 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:38663 (local address 127.0.206.66:45775)
0619 10:17:31.849027 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0619 10:17:31.849036 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0619 10:17:31.849494 (+ 458us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 10:17:31.849721 (+ 227us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 10:17:31.849724 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 10:17:31.850050 (+ 326us) client_negotiation.cc:189] Negotiated authn=SASL
0619 10:17:31.850225 (+ 175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:31.850229 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:31.850967 (+ 738us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:31.850968 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:31.851700 (+ 732us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:31.851702 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:31.852048 (+ 346us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:31.852049 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:31.852174 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:31.852837 (+ 663us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 10:17:31.852848 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0619 10:17:31.856149 (+ 3301us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 10:17:31.857811 (+ 1662us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 10:17:31.857812 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 10:17:31.857812 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 10:17:31.858039 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 10:17:31.858263 (+ 224us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 10:17:31.858264 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 10:17:31.858264 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 10:17:31.858320 (+ 56us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 10:17:31.858860 (+ 540us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 10:17:31.858861 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 10:17:31.858902 (+ 41us) client_negotiation.cc:769] Sending connection context
0619 10:17:31.859681 (+ 779us) client_negotiation.cc:240] Negotiation successful
0619 10:17:31.859787 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":205,"thread_start_us":69,"threads_started":1}
I20260619 10:17:31.860355 8001 heartbeater.cc:347] Connected to a master server at 127.0.206.125:40043
I20260619 10:17:31.860455 8001 heartbeater.cc:464] Registering TS with master...
I20260619 10:17:31.860468 7866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:31.849027 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.66:44267 (local address 127.0.206.124:42647)
0619 10:17:31.849071 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0619 10:17:31.849073 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 10:17:31.849174 (+ 101us) server_negotiation.cc:407] Connection header received
0619 10:17:31.849602 (+ 428us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:17:31.849604 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:17:31.849620 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:17:31.849650 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:17:31.850332 (+ 682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:31.851092 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:31.852219 (+ 1127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:31.852527 (+ 308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:31.852589 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:31.857480 (+ 4891us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 10:17:31.857482 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 10:17:31.857482 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 10:17:31.857491 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0619 10:17:31.859157 (+ 1666us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:17:31.859558 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:17:31.859560 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:17:31.859560 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:17:31.859620 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:17:31.859825 (+ 205us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:17:31.859826 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:17:31.859826 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:17:31.859965 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 10:17:31.860039 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0619 10:17:31.860273 (+ 234us) server_negotiation.cc:299] Negotiation successful
0619 10:17:31.860383 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260619 10:17:31.860543 8000 heartbeater.cc:347] Connected to a master server at 127.0.206.126:38663
I20260619 10:17:31.860571 8000 heartbeater.cc:464] Registering TS with master...
I20260619 10:17:31.860638 8000 heartbeater.cc:511] Master 127.0.206.126:38663 requested a full tablet report, sending...
I20260619 10:17:31.860776 825 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTU9OP6/build/release/bin/kudu
/tmp/dist-test-taskTU9OP6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781864224577058-825-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781864224577058-825-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781864224577058-825-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781864224577058-825-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.206.67:43923
--local_ip_for_outbound_sockets=127.0.206.67
--tserver_master_addrs=127.0.206.126:38663
--webserver_port=37187
--webserver_interface=127.0.206.67
--keytab_file=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781864224577058-825-0/krb5kdc/oryx_127.0.206.67.keytab
--principal=oryx/127.0.206.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.206.84:33273
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.0.206.126:38663,127.0.206.125:40043
--tserver_master_addrs=127.0.206.126:38663,127.0.206.125:40043,127.0.206.124:42647 with env {KRB5CCNAME=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781864224577058-825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781864224577058-825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781864224577058-825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 10:17:31.860639 8001 heartbeater.cc:511] Master 127.0.206.125:40043 requested a full tablet report, sending...
I20260619 10:17:31.861115 6898 ts_manager.cc:194] Re-registered known tserver with Master: 04a062435b8049fa92d4f33097d4e8a2 (127.0.206.66:37571)
I20260619 10:17:31.861188 7497 ts_manager.cc:194] Registered new tserver with Master: 04a062435b8049fa92d4f33097d4e8a2 (127.0.206.66:37571)
I20260619 10:17:31.861586 6898 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.0.206.66@KRBTEST.COM'} at 127.0.206.66:45775
I20260619 10:17:31.861982 8005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:31.848980 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.124:42647 (local address 127.0.206.66:44267)
0619 10:17:31.849117 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0619 10:17:31.849126 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0619 10:17:31.849491 (+ 365us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 10:17:31.849722 (+ 231us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 10:17:31.849725 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 10:17:31.850153 (+ 428us) client_negotiation.cc:189] Negotiated authn=SASL
0619 10:17:31.850226 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:31.850236 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:31.851190 (+ 954us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:31.851190 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:31.852115 (+ 925us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:31.852118 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:31.852642 (+ 524us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:31.852643 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:31.852727 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:31.853755 (+ 1028us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 10:17:31.853760 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 10:17:31.857403 (+ 3643us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 10:17:31.859232 (+ 1829us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 10:17:31.859232 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 10:17:31.859233 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 10:17:31.859483 (+ 250us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 10:17:31.859695 (+ 212us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 10:17:31.859695 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 10:17:31.859696 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 10:17:31.859760 (+ 64us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 10:17:31.860030 (+ 270us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 10:17:31.860031 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 10:17:31.860076 (+ 45us) client_negotiation.cc:769] Sending connection context
0619 10:17:31.860695 (+ 619us) client_negotiation.cc:240] Negotiation successful
0619 10:17:31.860780 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"mutex_wait_us":56,"thread_start_us":68,"threads_started":1}
I20260619 10:17:31.862334 7999 heartbeater.cc:347] Connected to a master server at 127.0.206.124:42647
I20260619 10:17:31.862383 7999 heartbeater.cc:464] Registering TS with master...
I20260619 10:17:31.862447 7999 heartbeater.cc:511] Master 127.0.206.124:42647 requested a full tablet report, sending...
I20260619 10:17:31.862771 7683 ts_manager.cc:194] Registered new tserver with Master: 04a062435b8049fa92d4f33097d4e8a2 (127.0.206.66:37571)
W20260619 10:17:31.939414 8006 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 10:17:31.939615 8006 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 10:17:31.939648 8006 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 10:17:31.940789 8006 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 10:17:31.940847 8006 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 10:17:31.940891 8006 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.206.67
I20260619 10:17:31.942302 8006 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.206.84:33273
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781864224577058-825-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781864224577058-825-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781864224577058-825-0/krb5kdc/oryx_127.0.206.67.keytab
--principal=oryx/127.0.206.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.206.67:43923
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781864224577058-825-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.206.67
--webserver_port=37187
--tserver_master_addrs=127.0.206.126:38663,127.0.206.125:40043,127.0.206.124:42647
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8006
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.206.67
--log_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781864224577058-825-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 09:32:44 UTC on ccycloud.quasar-gbtldp.root.comops.site
build id 4455440
I20260619 10:17:31.942538 8006 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 10:17:31.942764 8006 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 10:17:31.947744 8012 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 10:17:31.947718 8014 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 10:17:31.947969 8011 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 10:17:31.947922 8006 server_base.cc:1034] running on GCE node
I20260619 10:17:31.948190 8006 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 10:17:31.948371 8006 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 10:17:31.949498 8006 hybrid_clock.cc:648] HybridClock initialized: now 1781864251949484 us; error 30 us; skew 500 ppm
Jun 19 10:17:31 dist-test-slave-6pt2 krb5kdc[6853](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864251, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.206.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 10:17:31.951797 8006 init.cc:377] Logged in from keytab as oryx/127.0.206.67@KRBTEST.COM (short username oryx)
I20260619 10:17:31.952227 8006 webserver.cc:460] Webserver started at http://127.0.206.67:37187/ using document root <none> and password file <none>
I20260619 10:17:31.952433 8006 fs_manager.cc:359] Metadata directory not provided
I20260619 10:17:31.952507 8006 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 10:17:31.953720 8006 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 10:17:31.954210 8021 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 10:17:31.954361 8006 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 10:17:31.954448 8006 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781864224577058-825-0/minicluster-data/ts-2/data,/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781864224577058-825-0/minicluster-data/ts-2/wal
uuid: "9c6d725a83a94c8da04ad1daed4d3816"
format_stamp: "Formatted at 2026-06-19 10:17:31 on dist-test-slave-6pt2"
server_key: "da5fcbd3c4224fa4bc075103d9d08618"
server_key_iv: "707865a767bc195eac029631578c9012"
server_key_version: "encryptionkey@0"
I20260619 10:17:31.954788 8006 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781864224577058-825-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781864224577058-825-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781864224577058-825-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 10:17:31.968010 8006 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 10:17:31.969647 8006 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 10:17:31.969789 8006 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 10:17:31.970005 8006 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 10:17:31.970273 8006 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 10:17:31.970321 8006 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 10:17:31.970355 8006 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 10:17:31.970381 8006 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 10:17:31.976313 8006 rpc_server.cc:307] RPC server started. Bound to: 127.0.206.67:43923
I20260619 10:17:31.976353 8134 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.206.67:43923 every 8 connection(s)
I20260619 10:17:31.976661 8006 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781864224577058-825-0/minicluster-data/ts-2/data/info.pb
Jun 19 10:17:31 dist-test-slave-6pt2 krb5kdc[6853](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864251, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.206.67@KRBTEST.COM for oryx/127.0.206.125@KRBTEST.COM
Jun 19 10:17:31 dist-test-slave-6pt2 krb5kdc[6853](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864251, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.206.67@KRBTEST.COM for oryx/127.0.206.126@KRBTEST.COM
Jun 19 10:17:31 dist-test-slave-6pt2 krb5kdc[6853](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864251, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.206.67@KRBTEST.COM for oryx/127.0.206.124@KRBTEST.COM
I20260619 10:17:31.986076 825 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTU9OP6/build/release/bin/kudu as pid 8006
I20260619 10:17:31.987361 6956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:31.977266 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.67:47711 (local address 127.0.206.126:38663)
0619 10:17:31.977327 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0619 10:17:31.977329 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 10:17:31.977511 (+ 182us) server_negotiation.cc:407] Connection header received
0619 10:17:31.978674 (+ 1163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:17:31.978676 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:17:31.978689 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:17:31.978742 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:17:31.979458 (+ 716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:31.980215 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:31.981081 (+ 866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:31.981385 (+ 304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:31.981449 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:31.984222 (+ 2773us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 10:17:31.984224 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 10:17:31.984224 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 10:17:31.984231 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 10:17:31.985826 (+ 1595us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:17:31.986274 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:17:31.986275 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:17:31.986276 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:17:31.986333 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:17:31.986646 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:17:31.986647 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:17:31.986648 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:17:31.986780 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 10:17:31.986825 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0619 10:17:31.987186 (+ 361us) server_negotiation.cc:299] Negotiation successful
0619 10:17:31.987267 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260619 10:17:31.987366 7548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:31.977704 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.67:59131 (local address 127.0.206.125:40043)
0619 10:17:31.977750 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0619 10:17:31.977752 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 10:17:31.977848 (+ 96us) server_negotiation.cc:407] Connection header received
0619 10:17:31.978793 (+ 945us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:17:31.978793 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:17:31.978802 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:17:31.978859 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:17:31.979437 (+ 578us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:31.979921 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:31.981253 (+ 1332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:31.981542 (+ 289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:31.981589 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:31.983754 (+ 2165us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 10:17:31.983755 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 10:17:31.983755 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 10:17:31.983761 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 10:17:31.985644 (+ 1883us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:17:31.986265 (+ 621us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:17:31.986266 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:17:31.986267 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:17:31.986331 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:17:31.986641 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:17:31.986642 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:17:31.986642 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:17:31.986788 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 10:17:31.986836 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0619 10:17:31.987186 (+ 350us) server_negotiation.cc:299] Negotiation successful
0619 10:17:31.987266 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260619 10:17:31.987650 7866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:31.977556 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.67:41053 (local address 127.0.206.124:42647)
0619 10:17:31.977607 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0619 10:17:31.977609 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 10:17:31.977648 (+ 39us) server_negotiation.cc:407] Connection header received
0619 10:17:31.978679 (+ 1031us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:17:31.978681 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:17:31.978694 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:17:31.978726 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:17:31.979466 (+ 740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:31.980194 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:31.981632 (+ 1438us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:31.981868 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:31.981927 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:31.984821 (+ 2894us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 10:17:31.984823 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 10:17:31.984823 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 10:17:31.984831 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 10:17:31.986454 (+ 1623us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:17:31.986839 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:17:31.986840 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:17:31.986840 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:17:31.986894 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:17:31.987082 (+ 188us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:17:31.987083 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:17:31.987083 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:17:31.987210 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 10:17:31.987260 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0619 10:17:31.987436 (+ 176us) server_negotiation.cc:299] Negotiation successful
0619 10:17:31.987517 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260619 10:17:31.987907 8139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:31.977270 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:38663 (local address 127.0.206.67:47711)
0619 10:17:31.977398 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0619 10:17:31.977427 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0619 10:17:31.978546 (+ 1119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 10:17:31.978808 (+ 262us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 10:17:31.978812 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 10:17:31.979204 (+ 392us) client_negotiation.cc:189] Negotiated authn=SASL
0619 10:17:31.979348 (+ 144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:31.979354 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:31.980326 (+ 972us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:31.980327 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:31.980993 (+ 666us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:31.980995 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:31.981458 (+ 463us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:31.981458 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:31.981551 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:31.982105 (+ 554us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 10:17:31.982115 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0619 10:17:31.984146 (+ 2031us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 10:17:31.985898 (+ 1752us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 10:17:31.985899 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 10:17:31.985899 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 10:17:31.986177 (+ 278us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 10:17:31.986406 (+ 229us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 10:17:31.986407 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 10:17:31.986407 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 10:17:31.986545 (+ 138us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 10:17:31.986871 (+ 326us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 10:17:31.986877 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 10:17:31.987023 (+ 146us) client_negotiation.cc:769] Sending connection context
0619 10:17:31.987496 (+ 473us) client_negotiation.cc:240] Negotiation successful
0619 10:17:31.987682 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":98,"mutex_wait_us":28,"thread_start_us":71,"threads_started":1}
I20260619 10:17:31.988076 8140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:31.977496 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.124:42647 (local address 127.0.206.67:41053)
0619 10:17:31.977589 (+ 93us) negotiation.cc:107] Waiting for socket to connect
0619 10:17:31.977599 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 10:17:31.978546 (+ 947us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 10:17:31.978808 (+ 262us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 10:17:31.978812 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 10:17:31.979219 (+ 407us) client_negotiation.cc:189] Negotiated authn=SASL
0619 10:17:31.979348 (+ 129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:31.979374 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:31.980282 (+ 908us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:31.980283 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:31.981181 (+ 898us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:31.981182 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:31.981960 (+ 778us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:31.981961 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:31.982022 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:31.982399 (+ 377us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 10:17:31.982403 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 10:17:31.984707 (+ 2304us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 10:17:31.986529 (+ 1822us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 10:17:31.986529 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 10:17:31.986530 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 10:17:31.986756 (+ 226us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 10:17:31.986960 (+ 204us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 10:17:31.986961 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 10:17:31.986961 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 10:17:31.987011 (+ 50us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 10:17:31.987292 (+ 281us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 10:17:31.987293 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 10:17:31.987327 (+ 34us) client_negotiation.cc:769] Sending connection context
0619 10:17:31.987809 (+ 482us) client_negotiation.cc:240] Negotiation successful
0619 10:17:31.987927 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":78,"mutex_wait_us":48,"thread_start_us":55,"threads_started":1}
I20260619 10:17:31.988300 8136 heartbeater.cc:347] Connected to a master server at 127.0.206.126:38663
I20260619 10:17:31.988370 8136 heartbeater.cc:464] Registering TS with master...
I20260619 10:17:31.988538 8136 heartbeater.cc:511] Master 127.0.206.126:38663 requested a full tablet report, sending...
I20260619 10:17:31.988822 8141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:31.977642 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.125:40043 (local address 127.0.206.67:59131)
0619 10:17:31.977781 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0619 10:17:31.977789 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0619 10:17:31.978553 (+ 764us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 10:17:31.978863 (+ 310us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 10:17:31.978864 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 10:17:31.979175 (+ 311us) client_negotiation.cc:189] Negotiated authn=SASL
0619 10:17:31.979348 (+ 173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:31.979352 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:31.980013 (+ 661us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:31.980014 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:31.981167 (+ 1153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:31.981170 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:31.981601 (+ 431us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:31.981602 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:31.981655 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:31.982240 (+ 585us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 10:17:31.982245 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 10:17:31.983673 (+ 1428us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 10:17:31.985734 (+ 2061us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 10:17:31.985740 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 10:17:31.985742 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 10:17:31.986123 (+ 381us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 10:17:31.986413 (+ 290us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 10:17:31.986414 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 10:17:31.986414 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 10:17:31.986546 (+ 132us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 10:17:31.986889 (+ 343us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 10:17:31.986890 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 10:17:31.987023 (+ 133us) client_negotiation.cc:769] Sending connection context
0619 10:17:31.988545 (+ 1522us) client_negotiation.cc:240] Negotiation successful
0619 10:17:31.988684 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"mutex_wait_us":3,"thread_start_us":58,"threads_started":1}
I20260619 10:17:31.988920 6898 ts_manager.cc:194] Re-registered known tserver with Master: 9c6d725a83a94c8da04ad1daed4d3816 (127.0.206.67:43923)
I20260619 10:17:31.989399 6898 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.0.206.67@KRBTEST.COM'} at 127.0.206.67:47711
I20260619 10:17:31.989771 8135 heartbeater.cc:347] Connected to a master server at 127.0.206.124:42647
I20260619 10:17:31.989821 8135 heartbeater.cc:464] Registering TS with master...
I20260619 10:17:31.989886 8135 heartbeater.cc:511] Master 127.0.206.124:42647 requested a full tablet report, sending...
I20260619 10:17:31.989926 8137 heartbeater.cc:347] Connected to a master server at 127.0.206.125:40043
I20260619 10:17:31.989988 8137 heartbeater.cc:464] Registering TS with master...
I20260619 10:17:31.990059 8137 heartbeater.cc:511] Master 127.0.206.125:40043 requested a full tablet report, sending...
I20260619 10:17:31.990162 7683 ts_manager.cc:194] Registered new tserver with Master: 9c6d725a83a94c8da04ad1daed4d3816 (127.0.206.67:43923)
I20260619 10:17:31.990315 7497 ts_manager.cc:194] Registered new tserver with Master: 9c6d725a83a94c8da04ad1daed4d3816 (127.0.206.67:43923)
I20260619 10:17:31.996485 7548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:31.988831 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48780 (local address 127.0.206.125:40043)
0619 10:17:31.988867 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0619 10:17:31.988869 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 10:17:31.989359 (+ 490us) server_negotiation.cc:407] Connection header received
0619 10:17:31.989451 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:17:31.989451 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:17:31.989460 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:17:31.989487 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:17:31.990559 (+ 1072us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:31.991181 (+ 622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:31.992201 (+ 1020us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:31.992482 (+ 281us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:31.992541 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:31.993871 (+ 1330us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 10:17:31.993872 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 10:17:31.993872 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 10:17:31.993878 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 10:17:31.995442 (+ 1564us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:17:31.995797 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:17:31.995798 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:17:31.995798 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:17:31.995852 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:17:31.996029 (+ 177us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:17:31.996030 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:17:31.996030 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:17:31.996148 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 10:17:31.996240 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0619 10:17:31.996329 (+ 89us) server_negotiation.cc:299] Negotiation successful
0619 10:17:31.996407 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260619 10:17:31.996981 6956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:31.987216 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37440 (local address 127.0.206.126:38663)
0619 10:17:31.987437 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0619 10:17:31.987439 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 10:17:31.989064 (+ 1625us) server_negotiation.cc:407] Connection header received
0619 10:17:31.989148 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:17:31.989148 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:17:31.989157 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:17:31.989208 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:17:31.990889 (+ 1681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:31.991384 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:31.992143 (+ 759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:31.992428 (+ 285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:31.992484 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:31.994043 (+ 1559us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 10:17:31.994045 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 10:17:31.994045 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 10:17:31.994051 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 10:17:31.995580 (+ 1529us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:17:31.996162 (+ 582us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:17:31.996163 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:17:31.996164 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:17:31.996213 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:17:31.996410 (+ 197us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:17:31.996410 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:17:31.996411 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:17:31.996521 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 10:17:31.996609 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0619 10:17:31.996813 (+ 204us) server_negotiation.cc:299] Negotiation successful
0619 10:17:31.996885 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":207,"thread_start_us":78,"threads_started":1}
I20260619 10:17:31.997692 7866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:31.989111 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42080 (local address 127.0.206.124:42647)
0619 10:17:31.989146 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0619 10:17:31.989148 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 10:17:31.989417 (+ 269us) server_negotiation.cc:407] Connection header received
0619 10:17:31.989525 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:17:31.989526 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:17:31.989536 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:17:31.989566 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:17:31.990851 (+ 1285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:31.991443 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:31.992634 (+ 1191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:31.993311 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:31.993386 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:31.995007 (+ 1621us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 10:17:31.995008 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 10:17:31.995009 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 10:17:31.995015 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 10:17:31.996598 (+ 1583us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:17:31.996928 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:17:31.996929 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:17:31.996929 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:17:31.996979 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:17:31.997197 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:17:31.997198 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:17:31.997198 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:17:31.997319 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 10:17:31.997413 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0619 10:17:31.997516 (+ 103us) server_negotiation.cc:299] Negotiation successful
0619 10:17:31.997607 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260619 10:17:31.998708 6898 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37440:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 10:17:32.000197 6898 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 10:17:32.007262 7456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:32.003548 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.67:43923 (local address 127.0.0.1:35390)
0619 10:17:32.003593 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0619 10:17:32.003602 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0619 10:17:32.003635 (+ 33us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 10:17:32.003961 (+ 326us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 10:17:32.003962 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 10:17:32.003963 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 10:17:32.003996 (+ 33us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:32.003998 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:32.005054 (+ 1056us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:32.005055 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:32.006112 (+ 1057us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:32.006114 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:32.007003 (+ 889us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:32.007004 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:32.007069 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:32.007073 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 10:17:32.007143 (+ 70us) client_negotiation.cc:240] Negotiation successful
0619 10:17:32.007180 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":31}
I20260619 10:17:32.007370 8154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:32.003567 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.65:33905 (local address 127.0.0.1:38730)
0619 10:17:32.003676 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0619 10:17:32.003686 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 10:17:32.003735 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 10:17:32.004218 (+ 483us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 10:17:32.004219 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 10:17:32.004220 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 10:17:32.004255 (+ 35us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:32.004256 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:32.005529 (+ 1273us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:32.005530 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:32.006261 (+ 731us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:32.006262 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:32.007125 (+ 863us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:32.007126 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:32.007183 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:32.007187 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 10:17:32.007248 (+ 61us) client_negotiation.cc:240] Negotiation successful
0619 10:17:32.007282 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":89,"thread_start_us":73,"threads_started":1}
I20260619 10:17:32.007404 8156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:32.003849 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38730 (local address 127.0.206.65:33905)
0619 10:17:32.003999 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0619 10:17:32.004003 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 10:17:32.004013 (+ 10us) server_negotiation.cc:407] Connection header received
0619 10:17:32.004051 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:17:32.004053 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:17:32.004069 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:17:32.004148 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 10:17:32.004336 (+ 188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:32.005453 (+ 1117us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:32.006373 (+ 920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:32.007024 (+ 651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:32.007121 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:32.007197 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0619 10:17:32.007262 (+ 65us) server_negotiation.cc:299] Negotiation successful
0619 10:17:32.007308 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":117,"thread_start_us":84,"threads_started":1}
I20260619 10:17:32.007740 8155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:32.003694 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35390 (local address 127.0.206.67:43923)
0619 10:17:32.003813 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0619 10:17:32.003816 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 10:17:32.003823 (+ 7us) server_negotiation.cc:407] Connection header received
0619 10:17:32.003847 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:17:32.003848 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:17:32.003874 (+ 26us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:17:32.003951 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 10:17:32.004045 (+ 94us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:32.004964 (+ 919us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:32.006206 (+ 1242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:32.006916 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:32.006989 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:32.007054 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0619 10:17:32.007638 (+ 584us) server_negotiation.cc:299] Negotiation successful
0619 10:17:32.007673 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":98,"thread_start_us":73,"threads_started":1}
I20260619 10:17:32.008064 7796 tablet_service.cc:1467] Processing CreateTablet for tablet 45befbf01688436daf5063718b470826 (DEFAULT_TABLE table=test-table [id=7a81f4fecdf548ad9f72201f57f6444d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 10:17:32.008397 8157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:32.004131 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.66:37571 (local address 127.0.0.1:59078)
0619 10:17:32.004660 (+ 529us) negotiation.cc:107] Waiting for socket to connect
0619 10:17:32.004671 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 10:17:32.004724 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 10:17:32.004881 (+ 157us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 10:17:32.004881 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 10:17:32.004883 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 10:17:32.004917 (+ 34us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:32.004918 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:32.006119 (+ 1201us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:32.006120 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:32.007282 (+ 1162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:32.007284 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:32.008168 (+ 884us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:32.008169 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:32.008230 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:32.008234 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 10:17:32.008290 (+ 56us) client_negotiation.cc:240] Negotiation successful
0619 10:17:32.008329 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":512,"thread_start_us":69,"threads_started":1}
I20260619 10:17:32.008452 7796 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 45befbf01688436daf5063718b470826. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 10:17:32.008435 8158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:32.004261 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59078 (local address 127.0.206.66:37571)
0619 10:17:32.004420 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0619 10:17:32.004424 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 10:17:32.004726 (+ 302us) server_negotiation.cc:407] Connection header received
0619 10:17:32.004788 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:17:32.004790 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:17:32.004805 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:17:32.004871 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 10:17:32.004973 (+ 102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:32.005981 (+ 1008us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:32.007360 (+ 1379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:32.008063 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:32.008164 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:32.008230 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0619 10:17:32.008319 (+ 89us) server_negotiation.cc:299] Negotiation successful
0619 10:17:32.008355 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":129,"thread_start_us":96,"threads_started":1}
I20260619 10:17:32.008630 8069 tablet_service.cc:1467] Processing CreateTablet for tablet 45befbf01688436daf5063718b470826 (DEFAULT_TABLE table=test-table [id=7a81f4fecdf548ad9f72201f57f6444d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 10:17:32.008925 8069 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 45befbf01688436daf5063718b470826. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 10:17:32.012387 8159 tablet_bootstrap.cc:492] T 45befbf01688436daf5063718b470826 P 67adfa875ae04601b7b8fceb7610166c: Bootstrap starting.
I20260619 10:17:32.012977 8160 tablet_bootstrap.cc:492] T 45befbf01688436daf5063718b470826 P 9c6d725a83a94c8da04ad1daed4d3816: Bootstrap starting.
I20260619 10:17:32.013049 8159 tablet_bootstrap.cc:654] T 45befbf01688436daf5063718b470826 P 67adfa875ae04601b7b8fceb7610166c: Neither blocks nor log segments found. Creating new log.
I20260619 10:17:32.013053 7933 tablet_service.cc:1467] Processing CreateTablet for tablet 45befbf01688436daf5063718b470826 (DEFAULT_TABLE table=test-table [id=7a81f4fecdf548ad9f72201f57f6444d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 10:17:32.013343 7933 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 45befbf01688436daf5063718b470826. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 10:17:32.013339 8159 log.cc:824] T 45befbf01688436daf5063718b470826 P 67adfa875ae04601b7b8fceb7610166c: Log is configured to *not* fsync() on all Append() calls
I20260619 10:17:32.013477 8160 tablet_bootstrap.cc:654] T 45befbf01688436daf5063718b470826 P 9c6d725a83a94c8da04ad1daed4d3816: Neither blocks nor log segments found. Creating new log.
I20260619 10:17:32.013746 8160 log.cc:824] T 45befbf01688436daf5063718b470826 P 9c6d725a83a94c8da04ad1daed4d3816: Log is configured to *not* fsync() on all Append() calls
I20260619 10:17:32.014117 8159 tablet_bootstrap.cc:492] T 45befbf01688436daf5063718b470826 P 67adfa875ae04601b7b8fceb7610166c: No bootstrap required, opened a new log
I20260619 10:17:32.014189 8159 ts_tablet_manager.cc:1397] T 45befbf01688436daf5063718b470826 P 67adfa875ae04601b7b8fceb7610166c: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260619 10:17:32.014467 8160 tablet_bootstrap.cc:492] T 45befbf01688436daf5063718b470826 P 9c6d725a83a94c8da04ad1daed4d3816: No bootstrap required, opened a new log
I20260619 10:17:32.014518 8160 ts_tablet_manager.cc:1397] T 45befbf01688436daf5063718b470826 P 9c6d725a83a94c8da04ad1daed4d3816: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260619 10:17:32.015949 8163 tablet_bootstrap.cc:492] T 45befbf01688436daf5063718b470826 P 04a062435b8049fa92d4f33097d4e8a2: Bootstrap starting.
I20260619 10:17:32.016031 8159 raft_consensus.cc:348] T 45befbf01688436daf5063718b470826 P 67adfa875ae04601b7b8fceb7610166c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c6d725a83a94c8da04ad1daed4d3816" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 43923 } } peers { permanent_uuid: "67adfa875ae04601b7b8fceb7610166c" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 33905 } } peers { permanent_uuid: "04a062435b8049fa92d4f33097d4e8a2" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 37571 } }
I20260619 10:17:32.016180 8159 raft_consensus.cc:374] T 45befbf01688436daf5063718b470826 P 67adfa875ae04601b7b8fceb7610166c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 10:17:32.016214 8159 raft_consensus.cc:729] T 45befbf01688436daf5063718b470826 P 67adfa875ae04601b7b8fceb7610166c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 67adfa875ae04601b7b8fceb7610166c, State: Initialized, Role: FOLLOWER
I20260619 10:17:32.016252 8160 raft_consensus.cc:348] T 45befbf01688436daf5063718b470826 P 9c6d725a83a94c8da04ad1daed4d3816 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c6d725a83a94c8da04ad1daed4d3816" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 43923 } } peers { permanent_uuid: "67adfa875ae04601b7b8fceb7610166c" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 33905 } } peers { permanent_uuid: "04a062435b8049fa92d4f33097d4e8a2" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 37571 } }
I20260619 10:17:32.016345 8159 consensus_queue.cc:260] T 45befbf01688436daf5063718b470826 P 67adfa875ae04601b7b8fceb7610166c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c6d725a83a94c8da04ad1daed4d3816" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 43923 } } peers { permanent_uuid: "67adfa875ae04601b7b8fceb7610166c" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 33905 } } peers { permanent_uuid: "04a062435b8049fa92d4f33097d4e8a2" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 37571 } }
I20260619 10:17:32.016386 8160 raft_consensus.cc:374] T 45befbf01688436daf5063718b470826 P 9c6d725a83a94c8da04ad1daed4d3816 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 10:17:32.016417 8160 raft_consensus.cc:729] T 45befbf01688436daf5063718b470826 P 9c6d725a83a94c8da04ad1daed4d3816 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c6d725a83a94c8da04ad1daed4d3816, State: Initialized, Role: FOLLOWER
I20260619 10:17:32.016501 8163 tablet_bootstrap.cc:654] T 45befbf01688436daf5063718b470826 P 04a062435b8049fa92d4f33097d4e8a2: Neither blocks nor log segments found. Creating new log.
I20260619 10:17:32.016536 8160 consensus_queue.cc:260] T 45befbf01688436daf5063718b470826 P 9c6d725a83a94c8da04ad1daed4d3816 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c6d725a83a94c8da04ad1daed4d3816" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 43923 } } peers { permanent_uuid: "67adfa875ae04601b7b8fceb7610166c" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 33905 } } peers { permanent_uuid: "04a062435b8049fa92d4f33097d4e8a2" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 37571 } }
I20260619 10:17:32.016602 8159 ts_tablet_manager.cc:1428] T 45befbf01688436daf5063718b470826 P 67adfa875ae04601b7b8fceb7610166c: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260619 10:17:32.016710 7863 heartbeater.cc:503] Master 127.0.206.126:38663 was elected leader, sending a full tablet report...
I20260619 10:17:32.016752 8160 ts_tablet_manager.cc:1428] T 45befbf01688436daf5063718b470826 P 9c6d725a83a94c8da04ad1daed4d3816: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260619 10:17:32.016794 8163 log.cc:824] T 45befbf01688436daf5063718b470826 P 04a062435b8049fa92d4f33097d4e8a2: Log is configured to *not* fsync() on all Append() calls
I20260619 10:17:32.017047 8136 heartbeater.cc:503] Master 127.0.206.126:38663 was elected leader, sending a full tablet report...
I20260619 10:17:32.018078 8163 tablet_bootstrap.cc:492] T 45befbf01688436daf5063718b470826 P 04a062435b8049fa92d4f33097d4e8a2: No bootstrap required, opened a new log
I20260619 10:17:32.018138 8163 ts_tablet_manager.cc:1397] T 45befbf01688436daf5063718b470826 P 04a062435b8049fa92d4f33097d4e8a2: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260619 10:17:32.020303 8163 raft_consensus.cc:348] T 45befbf01688436daf5063718b470826 P 04a062435b8049fa92d4f33097d4e8a2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c6d725a83a94c8da04ad1daed4d3816" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 43923 } } peers { permanent_uuid: "67adfa875ae04601b7b8fceb7610166c" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 33905 } } peers { permanent_uuid: "04a062435b8049fa92d4f33097d4e8a2" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 37571 } }
I20260619 10:17:32.020447 8163 raft_consensus.cc:374] T 45befbf01688436daf5063718b470826 P 04a062435b8049fa92d4f33097d4e8a2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 10:17:32.020490 8163 raft_consensus.cc:729] T 45befbf01688436daf5063718b470826 P 04a062435b8049fa92d4f33097d4e8a2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 04a062435b8049fa92d4f33097d4e8a2, State: Initialized, Role: FOLLOWER
I20260619 10:17:32.020596 8163 consensus_queue.cc:260] T 45befbf01688436daf5063718b470826 P 04a062435b8049fa92d4f33097d4e8a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c6d725a83a94c8da04ad1daed4d3816" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 43923 } } peers { permanent_uuid: "67adfa875ae04601b7b8fceb7610166c" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 33905 } } peers { permanent_uuid: "04a062435b8049fa92d4f33097d4e8a2" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 37571 } }
I20260619 10:17:32.020820 8163 ts_tablet_manager.cc:1428] T 45befbf01688436daf5063718b470826 P 04a062435b8049fa92d4f33097d4e8a2: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260619 10:17:32.020885 8000 heartbeater.cc:503] Master 127.0.206.126:38663 was elected leader, sending a full tablet report...
I20260619 10:17:32.035389 7866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:32.033057 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42092 (local address 127.0.206.124:42647)
0619 10:17:32.033109 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0619 10:17:32.033111 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 10:17:32.033122 (+ 11us) server_negotiation.cc:407] Connection header received
0619 10:17:32.033172 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:17:32.033173 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:17:32.033212 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:17:32.033283 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 10:17:32.033374 (+ 91us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:32.033847 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:32.034737 (+ 890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:32.035131 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:32.035200 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:32.035245 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0619 10:17:32.035318 (+ 73us) server_negotiation.cc:299] Negotiation successful
0619 10:17:32.035342 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260619 10:17:32.035391 8157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:32.032997 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.124:42647 (local address 127.0.0.1:42092)
0619 10:17:32.033050 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0619 10:17:32.033072 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0619 10:17:32.033106 (+ 34us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 10:17:32.033285 (+ 179us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 10:17:32.033286 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 10:17:32.033287 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 10:17:32.033327 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:32.033329 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:32.033902 (+ 573us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:32.033902 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:32.034672 (+ 770us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:32.034675 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:32.035204 (+ 529us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:32.035205 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:32.035263 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:32.035266 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 10:17:32.035312 (+ 46us) client_negotiation.cc:240] Negotiation successful
0619 10:17:32.035343 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36}
I20260619 10:17:32.035758 7693 raft_consensus.cc:1203] T 00000000000000000000000000000000 P 4bac10aeb06347f2bfadf295a3417d44 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.9] Dedup: 1.7->[1.8-1.9]
I20260619 10:17:32.036562 7722 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4bac10aeb06347f2bfadf295a3417d44 [sys.catalog]: SysCatalogTable state changed. Reason: New leader bd146d3a022940a68274ddac0993c2e3. Latest consensus state: current_term: 1 leader_uuid: "bd146d3a022940a68274ddac0993c2e3" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "bd146d3a022940a68274ddac0993c2e3" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 38663 } } peers { permanent_uuid: "cd27214b90874d35b38271d228518b20" member_type: VOTER last_known_addr { host: "127.0.206.125" port: 40043 } attrs { promote: false } } peers { permanent_uuid: "4bac10aeb06347f2bfadf295a3417d44" member_type: NON_VOTER last_known_addr { host: "127.0.206.124" port: 42647 } attrs { promote: true } } }
I20260619 10:17:32.036652 7722 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4bac10aeb06347f2bfadf295a3417d44 [sys.catalog]: This master's current role is: LEARNER
W20260619 10:17:32.098539 8002 tablet.cc:2369] T 45befbf01688436daf5063718b470826 P 04a062435b8049fa92d4f33097d4e8a2: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 10:17:32.132084 8167 raft_consensus.cc:482] T 45befbf01688436daf5063718b470826 P 04a062435b8049fa92d4f33097d4e8a2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 10:17:32.132208 8167 raft_consensus.cc:504] T 45befbf01688436daf5063718b470826 P 04a062435b8049fa92d4f33097d4e8a2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c6d725a83a94c8da04ad1daed4d3816" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 43923 } } peers { permanent_uuid: "67adfa875ae04601b7b8fceb7610166c" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 33905 } } peers { permanent_uuid: "04a062435b8049fa92d4f33097d4e8a2" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 37571 } }
I20260619 10:17:32.132539 8167 leader_election.cc:288] T 45befbf01688436daf5063718b470826 P 04a062435b8049fa92d4f33097d4e8a2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9c6d725a83a94c8da04ad1daed4d3816 (127.0.206.67:43923), 67adfa875ae04601b7b8fceb7610166c (127.0.206.65:33905)
I20260619 10:17:32.135486 8156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:32.132848 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.66:55351 (local address 127.0.206.65:33905)
0619 10:17:32.132903 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0619 10:17:32.132905 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 10:17:32.132914 (+ 9us) server_negotiation.cc:407] Connection header received
0619 10:17:32.132944 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:17:32.132945 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:17:32.132955 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:17:32.133021 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 10:17:32.133211 (+ 190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:32.133736 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:32.134718 (+ 982us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:32.135124 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:32.135228 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:32.135237 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0619 10:17:32.135373 (+ 136us) server_negotiation.cc:299] Negotiation successful
0619 10:17:32.135416 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260619 10:17:32.135488 8005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:32.132712 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.65:33905 (local address 127.0.206.66:55351)
0619 10:17:32.132764 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0619 10:17:32.132778 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0619 10:17:32.132821 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 10:17:32.133047 (+ 226us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 10:17:32.133048 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 10:17:32.133050 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 10:17:32.133134 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:32.133136 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:32.133809 (+ 673us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:32.133810 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:32.134646 (+ 836us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:32.134648 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:32.135229 (+ 581us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:32.135230 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:32.135303 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:32.135307 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 10:17:32.135361 (+ 54us) client_negotiation.cc:240] Negotiation successful
0619 10:17:32.135391 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32,"mutex_wait_us":4}
I20260619 10:17:32.135607 8155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:32.132780 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.66:49489 (local address 127.0.206.67:43923)
0619 10:17:32.132830 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0619 10:17:32.132832 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 10:17:32.132994 (+ 162us) server_negotiation.cc:407] Connection header received
0619 10:17:32.133069 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:17:32.133070 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:17:32.133079 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:17:32.133142 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 10:17:32.133294 (+ 152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:32.133745 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:32.134770 (+ 1025us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:32.135227 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:32.135298 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:32.135305 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0619 10:17:32.135484 (+ 179us) server_negotiation.cc:299] Negotiation successful
0619 10:17:32.135526 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260619 10:17:32.135624 8004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:32.132724 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.67:43923 (local address 127.0.206.66:49489)
0619 10:17:32.132928 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0619 10:17:32.132938 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 10:17:32.132995 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 10:17:32.133149 (+ 154us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 10:17:32.133149 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 10:17:32.133150 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 10:17:32.133227 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:32.133229 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:32.133844 (+ 615us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:32.133845 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:32.134662 (+ 817us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:32.134664 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:32.135306 (+ 642us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:32.135307 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:32.135390 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:32.135393 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 10:17:32.135448 (+ 55us) client_negotiation.cc:240] Negotiation successful
0619 10:17:32.135482 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":185,"mutex_wait_us":146}
I20260619 10:17:32.136044 7816 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "45befbf01688436daf5063718b470826" candidate_uuid: "04a062435b8049fa92d4f33097d4e8a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "67adfa875ae04601b7b8fceb7610166c" is_pre_election: true
I20260619 10:17:32.136204 7816 raft_consensus.cc:2454] T 45befbf01688436daf5063718b470826 P 67adfa875ae04601b7b8fceb7610166c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 04a062435b8049fa92d4f33097d4e8a2 in term 0.
I20260619 10:17:32.136432 7889 leader_election.cc:302] T 45befbf01688436daf5063718b470826 P 04a062435b8049fa92d4f33097d4e8a2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 04a062435b8049fa92d4f33097d4e8a2, 67adfa875ae04601b7b8fceb7610166c; no voters:
I20260619 10:17:32.136485 8089 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "45befbf01688436daf5063718b470826" candidate_uuid: "04a062435b8049fa92d4f33097d4e8a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9c6d725a83a94c8da04ad1daed4d3816" is_pre_election: true
I20260619 10:17:32.136557 8167 raft_consensus.cc:2781] T 45befbf01688436daf5063718b470826 P 04a062435b8049fa92d4f33097d4e8a2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 10:17:32.136606 8167 raft_consensus.cc:482] T 45befbf01688436daf5063718b470826 P 04a062435b8049fa92d4f33097d4e8a2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 10:17:32.136631 8167 raft_consensus.cc:3037] T 45befbf01688436daf5063718b470826 P 04a062435b8049fa92d4f33097d4e8a2 [term 0 FOLLOWER]: Advancing to term 1
I20260619 10:17:32.136660 8089 raft_consensus.cc:2454] T 45befbf01688436daf5063718b470826 P 9c6d725a83a94c8da04ad1daed4d3816 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 04a062435b8049fa92d4f33097d4e8a2 in term 0.
I20260619 10:17:32.137377 8167 raft_consensus.cc:504] T 45befbf01688436daf5063718b470826 P 04a062435b8049fa92d4f33097d4e8a2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c6d725a83a94c8da04ad1daed4d3816" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 43923 } } peers { permanent_uuid: "67adfa875ae04601b7b8fceb7610166c" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 33905 } } peers { permanent_uuid: "04a062435b8049fa92d4f33097d4e8a2" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 37571 } }
I20260619 10:17:32.137516 8167 leader_election.cc:288] T 45befbf01688436daf5063718b470826 P 04a062435b8049fa92d4f33097d4e8a2 [CANDIDATE]: Term 1 election: Requested vote from peers 9c6d725a83a94c8da04ad1daed4d3816 (127.0.206.67:43923), 67adfa875ae04601b7b8fceb7610166c (127.0.206.65:33905)
I20260619 10:17:32.137648 7816 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "45befbf01688436daf5063718b470826" candidate_uuid: "04a062435b8049fa92d4f33097d4e8a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "67adfa875ae04601b7b8fceb7610166c"
I20260619 10:17:32.137677 8089 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "45befbf01688436daf5063718b470826" candidate_uuid: "04a062435b8049fa92d4f33097d4e8a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9c6d725a83a94c8da04ad1daed4d3816"
I20260619 10:17:32.137720 7816 raft_consensus.cc:3037] T 45befbf01688436daf5063718b470826 P 67adfa875ae04601b7b8fceb7610166c [term 0 FOLLOWER]: Advancing to term 1
I20260619 10:17:32.137748 8089 raft_consensus.cc:3037] T 45befbf01688436daf5063718b470826 P 9c6d725a83a94c8da04ad1daed4d3816 [term 0 FOLLOWER]: Advancing to term 1
I20260619 10:17:32.138396 7816 raft_consensus.cc:2454] T 45befbf01688436daf5063718b470826 P 67adfa875ae04601b7b8fceb7610166c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 04a062435b8049fa92d4f33097d4e8a2 in term 1.
I20260619 10:17:32.138557 7889 leader_election.cc:302] T 45befbf01688436daf5063718b470826 P 04a062435b8049fa92d4f33097d4e8a2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 04a062435b8049fa92d4f33097d4e8a2, 67adfa875ae04601b7b8fceb7610166c; no voters:
I20260619 10:17:32.138684 8167 raft_consensus.cc:2781] T 45befbf01688436daf5063718b470826 P 04a062435b8049fa92d4f33097d4e8a2 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 10:17:32.138808 8167 raft_consensus.cc:686] T 45befbf01688436daf5063718b470826 P 04a062435b8049fa92d4f33097d4e8a2 [term 1 LEADER]: Becoming Leader. State: Replica: 04a062435b8049fa92d4f33097d4e8a2, State: Running, Role: LEADER
I20260619 10:17:32.138906 8089 raft_consensus.cc:2454] T 45befbf01688436daf5063718b470826 P 9c6d725a83a94c8da04ad1daed4d3816 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 04a062435b8049fa92d4f33097d4e8a2 in term 1.
I20260619 10:17:32.138875 8167 consensus_queue.cc:237] T 45befbf01688436daf5063718b470826 P 04a062435b8049fa92d4f33097d4e8a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c6d725a83a94c8da04ad1daed4d3816" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 43923 } } peers { permanent_uuid: "67adfa875ae04601b7b8fceb7610166c" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 33905 } } peers { permanent_uuid: "04a062435b8049fa92d4f33097d4e8a2" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 37571 } }
I20260619 10:17:32.139609 6898 catalog_manager.cc:5495] T 45befbf01688436daf5063718b470826 P 04a062435b8049fa92d4f33097d4e8a2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 04a062435b8049fa92d4f33097d4e8a2 (127.0.206.66). New cstate: current_term: 1 leader_uuid: "04a062435b8049fa92d4f33097d4e8a2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c6d725a83a94c8da04ad1daed4d3816" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 43923 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "67adfa875ae04601b7b8fceb7610166c" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 33905 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "04a062435b8049fa92d4f33097d4e8a2" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 37571 } health_report { overall_health: HEALTHY } } }
I20260619 10:17:32.140148 7459 sys_catalog.cc:455] T 00000000000000000000000000000000 P bd146d3a022940a68274ddac0993c2e3 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "bd146d3a022940a68274ddac0993c2e3" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "bd146d3a022940a68274ddac0993c2e3" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 38663 } } peers { permanent_uuid: "cd27214b90874d35b38271d228518b20" member_type: VOTER last_known_addr { host: "127.0.206.125" port: 40043 } attrs { promote: false } } peers { permanent_uuid: "4bac10aeb06347f2bfadf295a3417d44" member_type: NON_VOTER last_known_addr { host: "127.0.206.124" port: 42647 } attrs { promote: true } } }
I20260619 10:17:32.140240 7459 sys_catalog.cc:458] T 00000000000000000000000000000000 P bd146d3a022940a68274ddac0993c2e3 [sys.catalog]: This master's current role is: LEADER
I20260619 10:17:32.140911 7457 raft_consensus.cc:1050] T 00000000000000000000000000000000 P bd146d3a022940a68274ddac0993c2e3: attempting to promote NON_VOTER 4bac10aeb06347f2bfadf295a3417d44 to VOTER
I20260619 10:17:32.141031 7457 consensus_queue.cc:237] T 00000000000000000000000000000000 P bd146d3a022940a68274ddac0993c2e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 1.10, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "bd146d3a022940a68274ddac0993c2e3" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 38663 } } peers { permanent_uuid: "cd27214b90874d35b38271d228518b20" member_type: VOTER last_known_addr { host: "127.0.206.125" port: 40043 } attrs { promote: false } } peers { permanent_uuid: "4bac10aeb06347f2bfadf295a3417d44" member_type: VOTER last_known_addr { host: "127.0.206.124" port: 42647 } attrs { promote: false } }
I20260619 10:17:32.141530 7693 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 4bac10aeb06347f2bfadf295a3417d44 [term 1 LEARNER]: Refusing update from remote peer bd146d3a022940a68274ddac0993c2e3: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20260619 10:17:32.141551 7507 raft_consensus.cc:1261] T 00000000000000000000000000000000 P cd27214b90874d35b38271d228518b20 [term 1 FOLLOWER]: Refusing update from remote peer bd146d3a022940a68274ddac0993c2e3: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20260619 10:17:32.141734 8174 consensus_queue.cc:1035] T 00000000000000000000000000000000 P bd146d3a022940a68274ddac0993c2e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4bac10aeb06347f2bfadf295a3417d44" member_type: VOTER last_known_addr { host: "127.0.206.124" port: 42647 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260619 10:17:32.141835 7458 consensus_queue.cc:1035] T 00000000000000000000000000000000 P bd146d3a022940a68274ddac0993c2e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd27214b90874d35b38271d228518b20" member_type: VOTER last_known_addr { host: "127.0.206.125" port: 40043 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260619 10:17:32.142490 7457 raft_consensus.cc:2932] T 00000000000000000000000000000000 P bd146d3a022940a68274ddac0993c2e3 [term 1 LEADER]: Committing config change with OpId 1.11: config changed from index 7 to 11, 4bac10aeb06347f2bfadf295a3417d44 (127.0.206.124) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "bd146d3a022940a68274ddac0993c2e3" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 38663 } } peers { permanent_uuid: "cd27214b90874d35b38271d228518b20" member_type: VOTER last_known_addr { host: "127.0.206.125" port: 40043 } attrs { promote: false } } peers { permanent_uuid: "4bac10aeb06347f2bfadf295a3417d44" member_type: VOTER last_known_addr { host: "127.0.206.124" port: 42647 } attrs { promote: false } } }
I20260619 10:17:32.142656 7507 raft_consensus.cc:2932] T 00000000000000000000000000000000 P cd27214b90874d35b38271d228518b20 [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 7 to 11, 4bac10aeb06347f2bfadf295a3417d44 (127.0.206.124) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "bd146d3a022940a68274ddac0993c2e3" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 38663 } } peers { permanent_uuid: "cd27214b90874d35b38271d228518b20" member_type: VOTER last_known_addr { host: "127.0.206.125" port: 40043 } attrs { promote: false } } peers { permanent_uuid: "4bac10aeb06347f2bfadf295a3417d44" member_type: VOTER last_known_addr { host: "127.0.206.124" port: 42647 } attrs { promote: false } } }
I20260619 10:17:32.143307 7457 sys_catalog.cc:455] T 00000000000000000000000000000000 P bd146d3a022940a68274ddac0993c2e3 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "bd146d3a022940a68274ddac0993c2e3" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "bd146d3a022940a68274ddac0993c2e3" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 38663 } } peers { permanent_uuid: "cd27214b90874d35b38271d228518b20" member_type: VOTER last_known_addr { host: "127.0.206.125" port: 40043 } attrs { promote: false } } peers { permanent_uuid: "4bac10aeb06347f2bfadf295a3417d44" member_type: VOTER last_known_addr { host: "127.0.206.124" port: 42647 } attrs { promote: false } } }
I20260619 10:17:32.143466 7457 sys_catalog.cc:458] T 00000000000000000000000000000000 P bd146d3a022940a68274ddac0993c2e3 [sys.catalog]: This master's current role is: LEADER
I20260619 10:17:32.143532 8174 sys_catalog.cc:455] T 00000000000000000000000000000000 P bd146d3a022940a68274ddac0993c2e3 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "bd146d3a022940a68274ddac0993c2e3" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "bd146d3a022940a68274ddac0993c2e3" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 38663 } } peers { permanent_uuid: "cd27214b90874d35b38271d228518b20" member_type: VOTER last_known_addr { host: "127.0.206.125" port: 40043 } attrs { promote: false } } peers { permanent_uuid: "4bac10aeb06347f2bfadf295a3417d44" member_type: VOTER last_known_addr { host: "127.0.206.124" port: 42647 } attrs { promote: false } } }
I20260619 10:17:32.143592 8174 sys_catalog.cc:458] T 00000000000000000000000000000000 P bd146d3a022940a68274ddac0993c2e3 [sys.catalog]: This master's current role is: LEADER
I20260619 10:17:32.143671 7458 sys_catalog.cc:455] T 00000000000000000000000000000000 P bd146d3a022940a68274ddac0993c2e3 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "bd146d3a022940a68274ddac0993c2e3" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "bd146d3a022940a68274ddac0993c2e3" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 38663 } } peers { permanent_uuid: "cd27214b90874d35b38271d228518b20" member_type: VOTER last_known_addr { host: "127.0.206.125" port: 40043 } attrs { promote: false } } peers { permanent_uuid: "4bac10aeb06347f2bfadf295a3417d44" member_type: VOTER last_known_addr { host: "127.0.206.124" port: 42647 } attrs { promote: false } } }
I20260619 10:17:32.143734 7458 sys_catalog.cc:458] T 00000000000000000000000000000000 P bd146d3a022940a68274ddac0993c2e3 [sys.catalog]: This master's current role is: LEADER
I20260619 10:17:32.143613 7693 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 4bac10aeb06347f2bfadf295a3417d44 [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 7 to 11, 4bac10aeb06347f2bfadf295a3417d44 (127.0.206.124) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "bd146d3a022940a68274ddac0993c2e3" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 38663 } } peers { permanent_uuid: "cd27214b90874d35b38271d228518b20" member_type: VOTER last_known_addr { host: "127.0.206.125" port: 40043 } attrs { promote: false } } peers { permanent_uuid: "4bac10aeb06347f2bfadf295a3417d44" member_type: VOTER last_known_addr { host: "127.0.206.124" port: 42647 } attrs { promote: false } } }
I20260619 10:17:32.144639 7722 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4bac10aeb06347f2bfadf295a3417d44 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "bd146d3a022940a68274ddac0993c2e3" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "bd146d3a022940a68274ddac0993c2e3" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 38663 } } peers { permanent_uuid: "cd27214b90874d35b38271d228518b20" member_type: VOTER last_known_addr { host: "127.0.206.125" port: 40043 } attrs { promote: false } } peers { permanent_uuid: "4bac10aeb06347f2bfadf295a3417d44" member_type: VOTER last_known_addr { host: "127.0.206.124" port: 42647 } attrs { promote: false } } }
I20260619 10:17:32.144670 8176 sys_catalog.cc:455] T 00000000000000000000000000000000 P cd27214b90874d35b38271d228518b20 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "bd146d3a022940a68274ddac0993c2e3" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "bd146d3a022940a68274ddac0993c2e3" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 38663 } } peers { permanent_uuid: "cd27214b90874d35b38271d228518b20" member_type: VOTER last_known_addr { host: "127.0.206.125" port: 40043 } attrs { promote: false } } peers { permanent_uuid: "4bac10aeb06347f2bfadf295a3417d44" member_type: VOTER last_known_addr { host: "127.0.206.124" port: 42647 } attrs { promote: false } } }
I20260619 10:17:32.144734 7722 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4bac10aeb06347f2bfadf295a3417d44 [sys.catalog]: This master's current role is: FOLLOWER
I20260619 10:17:32.144752 8176 sys_catalog.cc:458] T 00000000000000000000000000000000 P cd27214b90874d35b38271d228518b20 [sys.catalog]: This master's current role is: FOLLOWER
I20260619 10:17:32.152184 8158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:32.150161 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59088 (local address 127.0.206.66:37571)
0619 10:17:32.150212 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0619 10:17:32.150214 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 10:17:32.150220 (+ 6us) server_negotiation.cc:407] Connection header received
0619 10:17:32.150266 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:17:32.150267 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:17:32.150280 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:17:32.150333 (+ 53us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 10:17:32.150414 (+ 81us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:32.150927 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:32.151596 (+ 669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:32.151766 (+ 170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:32.151813 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:32.151929 (+ 116us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 10:17:32.152013 (+ 84us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 10:17:32.152068 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0619 10:17:32.152082 (+ 14us) server_negotiation.cc:299] Negotiation successful
0619 10:17:32.152096 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260619 10:17:32.153695 8089 raft_consensus.cc:1261] T 45befbf01688436daf5063718b470826 P 9c6d725a83a94c8da04ad1daed4d3816 [term 1 FOLLOWER]: Refusing update from remote peer 04a062435b8049fa92d4f33097d4e8a2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 10:17:32.153717 7816 raft_consensus.cc:1261] T 45befbf01688436daf5063718b470826 P 67adfa875ae04601b7b8fceb7610166c [term 1 FOLLOWER]: Refusing update from remote peer 04a062435b8049fa92d4f33097d4e8a2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 10:17:32.153967 8167 consensus_queue.cc:1035] T 45befbf01688436daf5063718b470826 P 04a062435b8049fa92d4f33097d4e8a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9c6d725a83a94c8da04ad1daed4d3816" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 43923 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 10:17:32.154090 8172 consensus_queue.cc:1035] T 45befbf01688436daf5063718b470826 P 04a062435b8049fa92d4f33097d4e8a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "67adfa875ae04601b7b8fceb7610166c" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 33905 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 10:17:32.157195 8180 mvcc.cc:205] Tried to move back new op lower bound from 7298515976819277824 to 7298515976761135104. Current Snapshot: MvccSnapshot[applied={T|T < 7298515976819277824}]
I20260619 10:17:32.157594 8179 mvcc.cc:205] Tried to move back new op lower bound from 7298515976819277824 to 7298515976761135104. Current Snapshot: MvccSnapshot[applied={T|T < 7298515976819277824}]
I20260619 10:17:32.159818 6897 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37440:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 10:17:32.159888 6897 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37440:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 10:17:32.161233 6897 catalog_manager.cc:5782] T 00000000000000000000000000000000 P bd146d3a022940a68274ddac0993c2e3: Sending DeleteTablet for 3 replicas of tablet 45befbf01688436daf5063718b470826
I20260619 10:17:32.161501 7796 tablet_service.cc:1514] Processing DeleteTablet for tablet 45befbf01688436daf5063718b470826 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 10:17:32 UTC) from {username='oryx', principal='oryx/127.0.206.126@KRBTEST.COM'} at 127.0.0.1:38730
I20260619 10:17:32.161546 8069 tablet_service.cc:1514] Processing DeleteTablet for tablet 45befbf01688436daf5063718b470826 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 10:17:32 UTC) from {username='oryx', principal='oryx/127.0.206.126@KRBTEST.COM'} at 127.0.0.1:35390
I20260619 10:17:32.161782 8187 tablet_replica.cc:330] T 45befbf01688436daf5063718b470826 P 67adfa875ae04601b7b8fceb7610166c: stopping tablet replica
I20260619 10:17:32.161885 8187 raft_consensus.cc:2229] T 45befbf01688436daf5063718b470826 P 67adfa875ae04601b7b8fceb7610166c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 10:17:32.161988 8187 raft_consensus.cc:2258] T 45befbf01688436daf5063718b470826 P 67adfa875ae04601b7b8fceb7610166c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 10:17:32.163039 7933 tablet_service.cc:1514] Processing DeleteTablet for tablet 45befbf01688436daf5063718b470826 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 10:17:32 UTC) from {username='oryx', principal='oryx/127.0.206.126@KRBTEST.COM'} at 127.0.0.1:59078
I20260619 10:17:32.163131 825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTU9OP6/build/release/bin/kudu with pid 6866
I20260619 10:17:32.163642 8190 tablet_replica.cc:330] T 45befbf01688436daf5063718b470826 P 04a062435b8049fa92d4f33097d4e8a2: stopping tablet replica
I20260619 10:17:32.163775 8190 raft_consensus.cc:2229] T 45befbf01688436daf5063718b470826 P 04a062435b8049fa92d4f33097d4e8a2 [term 1 LEADER]: Raft consensus shutting down.
I20260619 10:17:32.163910 8190 raft_consensus.cc:2258] T 45befbf01688436daf5063718b470826 P 04a062435b8049fa92d4f33097d4e8a2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 10:17:32.164361 8190 ts_tablet_manager.cc:1905] T 45befbf01688436daf5063718b470826 P 04a062435b8049fa92d4f33097d4e8a2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 10:17:32.165987 8190 ts_tablet_manager.cc:1918] T 45befbf01688436daf5063718b470826 P 04a062435b8049fa92d4f33097d4e8a2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 10:17:32.166514 8190 log.cc:1196] T 45befbf01688436daf5063718b470826 P 04a062435b8049fa92d4f33097d4e8a2: Deleting WAL directory at /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781864224577058-825-0/minicluster-data/ts-1/wal/wals/45befbf01688436daf5063718b470826
I20260619 10:17:32.166898 8190 ts_tablet_manager.cc:1939] T 45befbf01688436daf5063718b470826 P 04a062435b8049fa92d4f33097d4e8a2: Deleting consensus metadata
I20260619 10:17:32.170840 7866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:32.162454 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42106 (local address 127.0.206.124:42647)
0619 10:17:32.162494 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0619 10:17:32.162496 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 10:17:32.162503 (+ 7us) server_negotiation.cc:407] Connection header received
0619 10:17:32.162521 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:17:32.162522 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:17:32.162531 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:17:32.162588 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:17:32.163570 (+ 982us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:32.164333 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:32.165481 (+ 1148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:32.165818 (+ 337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:32.165874 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:32.167554 (+ 1680us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 10:17:32.167556 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 10:17:32.167556 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 10:17:32.167565 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0619 10:17:32.169297 (+ 1732us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:17:32.169682 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:17:32.169684 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:17:32.169684 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:17:32.169745 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:17:32.169959 (+ 214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:17:32.169960 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:17:32.169960 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:17:32.170094 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 10:17:32.170192 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0619 10:17:32.170667 (+ 475us) server_negotiation.cc:299] Negotiation successful
0619 10:17:32.170742 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260619 10:17:32.171362 8187 ts_tablet_manager.cc:1905] T 45befbf01688436daf5063718b470826 P 67adfa875ae04601b7b8fceb7610166c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 10:17:32.173056 8187 ts_tablet_manager.cc:1918] T 45befbf01688436daf5063718b470826 P 67adfa875ae04601b7b8fceb7610166c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 10:17:32.173122 8187 log.cc:1196] T 45befbf01688436daf5063718b470826 P 67adfa875ae04601b7b8fceb7610166c: Deleting WAL directory at /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781864224577058-825-0/minicluster-data/ts-0/wal/wals/45befbf01688436daf5063718b470826
I20260619 10:17:32.173324 8187 ts_tablet_manager.cc:1939] T 45befbf01688436daf5063718b470826 P 67adfa875ae04601b7b8fceb7610166c: Deleting consensus metadata
W20260619 10:17:32.173615 7751 connection.cc:441] server connection from 127.0.0.1:38730 torn down before Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38730 (request call id 1) could send its response
I20260619 10:17:32.173668 8188 tablet_replica.cc:330] T 45befbf01688436daf5063718b470826 P 9c6d725a83a94c8da04ad1daed4d3816: stopping tablet replica
I20260619 10:17:32.173766 8188 raft_consensus.cc:2229] T 45befbf01688436daf5063718b470826 P 9c6d725a83a94c8da04ad1daed4d3816 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 10:17:32.173842 8188 raft_consensus.cc:2258] T 45befbf01688436daf5063718b470826 P 9c6d725a83a94c8da04ad1daed4d3816 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 10:17:32.174202 8188 ts_tablet_manager.cc:1905] T 45befbf01688436daf5063718b470826 P 9c6d725a83a94c8da04ad1daed4d3816: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 10:17:32.175614 8188 ts_tablet_manager.cc:1918] T 45befbf01688436daf5063718b470826 P 9c6d725a83a94c8da04ad1daed4d3816: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 10:17:32.175681 8188 log.cc:1196] T 45befbf01688436daf5063718b470826 P 9c6d725a83a94c8da04ad1daed4d3816: Deleting WAL directory at /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781864224577058-825-0/minicluster-data/ts-2/wal/wals/45befbf01688436daf5063718b470826
I20260619 10:17:32.175920 8188 ts_tablet_manager.cc:1939] T 45befbf01688436daf5063718b470826 P 9c6d725a83a94c8da04ad1daed4d3816: Deleting consensus metadata
W20260619 10:17:32.176239 8023 connection.cc:441] server connection from 127.0.0.1:35390 torn down before Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:35390 (request call id 1) could send its response
I20260619 10:17:32.180846 7548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:32.172236 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48788 (local address 127.0.206.125:40043)
0619 10:17:32.172316 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0619 10:17:32.172318 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 10:17:32.172325 (+ 7us) server_negotiation.cc:407] Connection header received
0619 10:17:32.172346 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:17:32.172347 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:17:32.172356 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:17:32.172413 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:17:32.173084 (+ 671us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:32.173868 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:32.175632 (+ 1764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:32.175968 (+ 336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:32.176017 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:32.177308 (+ 1291us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 10:17:32.177310 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 10:17:32.177310 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 10:17:32.177316 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 10:17:32.178835 (+ 1519us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:17:32.179507 (+ 672us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:17:32.179509 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:17:32.179509 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:17:32.179568 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:17:32.179785 (+ 217us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:17:32.179786 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:17:32.179786 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:17:32.179930 (+ 144us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 10:17:32.180025 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0619 10:17:32.180651 (+ 626us) server_negotiation.cc:299] Negotiation successful
0619 10:17:32.180726 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260619 10:17:32.181649 7866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:32.172276 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42122 (local address 127.0.206.124:42647)
0619 10:17:32.172457 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0619 10:17:32.172458 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 10:17:32.174762 (+ 2304us) server_negotiation.cc:407] Connection header received
0619 10:17:32.174850 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:17:32.174851 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:17:32.174860 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:17:32.174887 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:17:32.175560 (+ 673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:32.176315 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:32.177378 (+ 1063us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:32.177683 (+ 305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:32.177720 (+ 37us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:32.179005 (+ 1285us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 10:17:32.179006 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 10:17:32.179007 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 10:17:32.179013 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 10:17:32.180434 (+ 1421us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:17:32.180862 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:17:32.180863 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:17:32.180864 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:17:32.180917 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:17:32.181229 (+ 312us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:17:32.181230 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:17:32.181230 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:17:32.181341 (+ 111us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 10:17:32.181413 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0619 10:17:32.181541 (+ 128us) server_negotiation.cc:299] Negotiation successful
0619 10:17:32.181593 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":161}
I20260619 10:17:33.158699 8198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:33.158437 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:38663 (local address 127.0.206.67:35983)
0619 10:17:33.158539 (+ 102us) negotiation.cc:107] Waiting for socket to connect
0619 10:17:33.158646 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.126:38663: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":85,"thread_start_us":62,"threads_started":1}
I20260619 10:17:33.158699 8197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:33.158303 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:38663 (local address 127.0.206.65:55825)
0619 10:17:33.158539 (+ 236us) negotiation.cc:107] Waiting for socket to connect
0619 10:17:33.158636 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.126:38663: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":100,"threads_started":1}
W20260619 10:17:33.159132 7863 heartbeater.cc:644] Failed to heartbeat to 127.0.206.126:38663 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.0.206.126:38663: connect: Connection refused (error 111)
W20260619 10:17:33.159154 8136 heartbeater.cc:644] Failed to heartbeat to 127.0.206.126:38663 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.0.206.126:38663: connect: Connection refused (error 111)
I20260619 10:17:33.159766 8199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:33.159439 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:38663 (local address 127.0.206.66:49371)
0619 10:17:33.159625 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0619 10:17:33.159733 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.126:38663: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":170,"thread_start_us":70,"threads_started":1}
W20260619 10:17:33.159960 8000 heartbeater.cc:644] Failed to heartbeat to 127.0.206.126:38663 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.0.206.126:38663: connect: Connection refused (error 111)
I20260619 10:17:33.661545 8200 raft_consensus.cc:482] T 00000000000000000000000000000000 P cd27214b90874d35b38271d228518b20 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader bd146d3a022940a68274ddac0993c2e3)
I20260619 10:17:33.661681 8200 raft_consensus.cc:504] T 00000000000000000000000000000000 P cd27214b90874d35b38271d228518b20 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "bd146d3a022940a68274ddac0993c2e3" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 38663 } } peers { permanent_uuid: "cd27214b90874d35b38271d228518b20" member_type: VOTER last_known_addr { host: "127.0.206.125" port: 40043 } attrs { promote: false } } peers { permanent_uuid: "4bac10aeb06347f2bfadf295a3417d44" member_type: VOTER last_known_addr { host: "127.0.206.124" port: 42647 } attrs { promote: false } }
I20260619 10:17:33.661794 8201 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4bac10aeb06347f2bfadf295a3417d44 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader bd146d3a022940a68274ddac0993c2e3)
I20260619 10:17:33.661990 8200 leader_election.cc:288] T 00000000000000000000000000000000 P cd27214b90874d35b38271d228518b20 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers bd146d3a022940a68274ddac0993c2e3 (127.0.206.126:38663), 4bac10aeb06347f2bfadf295a3417d44 (127.0.206.124:42647)
I20260619 10:17:33.661949 8201 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4bac10aeb06347f2bfadf295a3417d44 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "bd146d3a022940a68274ddac0993c2e3" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 38663 } } peers { permanent_uuid: "cd27214b90874d35b38271d228518b20" member_type: VOTER last_known_addr { host: "127.0.206.125" port: 40043 } attrs { promote: false } } peers { permanent_uuid: "4bac10aeb06347f2bfadf295a3417d44" member_type: VOTER last_known_addr { host: "127.0.206.124" port: 42647 } attrs { promote: false } }
I20260619 10:17:33.662192 8201 leader_election.cc:288] T 00000000000000000000000000000000 P 4bac10aeb06347f2bfadf295a3417d44 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers bd146d3a022940a68274ddac0993c2e3 (127.0.206.126:38663), cd27214b90874d35b38271d228518b20 (127.0.206.125:40043)
I20260619 10:17:33.662333 8202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:33.662079 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:38663 (local address 127.0.0.1:37944)
0619 10:17:33.662161 (+ 82us) negotiation.cc:107] Waiting for socket to connect
0619 10:17:33.662218 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.126:38663: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68,"thread_start_us":50,"threads_started":1}
I20260619 10:17:33.662493 8204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:33.662282 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:38663 (local address 127.0.0.1:37950)
0619 10:17:33.662400 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0619 10:17:33.662458 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.126:38663: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":55,"threads_started":1}
W20260619 10:17:33.662549 7483 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.206.126:38663: connect: Connection refused (error 111)
W20260619 10:17:33.662825 7669 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.206.126:38663: connect: Connection refused (error 111)
I20260619 10:17:33.662936 8202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:33.662851 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:38663 (local address 127.0.0.1:37966)
0619 10:17:33.662891 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0619 10:17:33.662918 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.126:38663: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
W20260619 10:17:33.663028 7483 leader_election.cc:334] T 00000000000000000000000000000000 P cd27214b90874d35b38271d228518b20 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer bd146d3a022940a68274ddac0993c2e3 (127.0.206.126:38663): Network error: Client connection negotiation failed: client connection to 127.0.206.126:38663: connect: Connection refused (error 111)
I20260619 10:17:33.663349 8205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:33.663251 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:38663 (local address 127.0.0.1:37982)
0619 10:17:33.663289 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0619 10:17:33.663321 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.126:38663: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
W20260619 10:17:33.663420 7669 leader_election.cc:334] T 00000000000000000000000000000000 P 4bac10aeb06347f2bfadf295a3417d44 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer bd146d3a022940a68274ddac0993c2e3 (127.0.206.126:38663): Network error: Client connection negotiation failed: client connection to 127.0.206.126:38663: connect: Connection refused (error 111)
I20260619 10:17:33.665416 8206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:33.662522 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42134 (local address 127.0.206.124:42647)
0619 10:17:33.662745 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0619 10:17:33.662746 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 10:17:33.662753 (+ 7us) server_negotiation.cc:407] Connection header received
0619 10:17:33.662772 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:17:33.662773 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:17:33.662783 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:17:33.662839 (+ 56us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 10:17:33.663103 (+ 264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:33.663761 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:33.664697 (+ 936us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:33.665092 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:33.665168 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:33.665179 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0619 10:17:33.665321 (+ 142us) server_negotiation.cc:299] Negotiation successful
0619 10:17:33.665359 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":210,"thread_start_us":39,"threads_started":1}
I20260619 10:17:33.665407 8203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:33.662244 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.124:42647 (local address 127.0.0.1:42134)
0619 10:17:33.662328 (+ 84us) negotiation.cc:107] Waiting for socket to connect
0619 10:17:33.662338 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 10:17:33.662388 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 10:17:33.662865 (+ 477us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 10:17:33.662866 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 10:17:33.662869 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 10:17:33.663004 (+ 135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:33.663007 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:33.663850 (+ 843us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:33.663851 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:33.664601 (+ 750us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:33.664603 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:33.665168 (+ 565us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:33.665169 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:33.665230 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:33.665233 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 10:17:33.665298 (+ 65us) client_negotiation.cc:240] Negotiation successful
0619 10:17:33.665342 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":70,"thread_start_us":39,"threads_started":1}
I20260619 10:17:33.665530 8208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:33.662658 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48790 (local address 127.0.206.125:40043)
0619 10:17:33.663030 (+ 372us) server_negotiation.cc:206] Beginning negotiation
0619 10:17:33.663032 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 10:17:33.663038 (+ 6us) server_negotiation.cc:407] Connection header received
0619 10:17:33.663058 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:17:33.663059 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:17:33.663068 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:17:33.663137 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 10:17:33.663301 (+ 164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:33.663859 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:33.664754 (+ 895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:33.665186 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:33.665271 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:33.665281 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0619 10:17:33.665411 (+ 130us) server_negotiation.cc:299] Negotiation successful
0619 10:17:33.665450 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":346,"thread_start_us":79,"threads_started":1}
I20260619 10:17:33.665673 8204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:33.662446 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.125:40043 (local address 127.0.0.1:48790)
0619 10:17:33.662563 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0619 10:17:33.662572 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0619 10:17:33.662640 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 10:17:33.663143 (+ 503us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 10:17:33.663145 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 10:17:33.663148 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 10:17:33.663212 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:33.663214 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:33.663935 (+ 721us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:33.663936 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:33.664653 (+ 717us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:33.664655 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:33.665273 (+ 618us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:33.665274 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:33.665333 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:33.665337 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 10:17:33.665404 (+ 67us) client_negotiation.cc:240] Negotiation successful
0619 10:17:33.665443 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":107,"mutex_wait_us":5,"thread_start_us":41,"threads_started":1}
I20260619 10:17:33.665905 7507 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "4bac10aeb06347f2bfadf295a3417d44" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "cd27214b90874d35b38271d228518b20" is_pre_election: true
I20260619 10:17:33.665997 7507 raft_consensus.cc:2454] T 00000000000000000000000000000000 P cd27214b90874d35b38271d228518b20 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4bac10aeb06347f2bfadf295a3417d44 in term 1.
I20260619 10:17:33.666007 7693 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "cd27214b90874d35b38271d228518b20" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "4bac10aeb06347f2bfadf295a3417d44" is_pre_election: true
I20260619 10:17:33.666102 7693 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 4bac10aeb06347f2bfadf295a3417d44 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd27214b90874d35b38271d228518b20 in term 1.
I20260619 10:17:33.666141 7669 leader_election.cc:302] T 00000000000000000000000000000000 P 4bac10aeb06347f2bfadf295a3417d44 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4bac10aeb06347f2bfadf295a3417d44, cd27214b90874d35b38271d228518b20; no voters: bd146d3a022940a68274ddac0993c2e3
I20260619 10:17:33.666245 8201 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4bac10aeb06347f2bfadf295a3417d44 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260619 10:17:33.666237 7483 leader_election.cc:302] T 00000000000000000000000000000000 P cd27214b90874d35b38271d228518b20 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4bac10aeb06347f2bfadf295a3417d44, cd27214b90874d35b38271d228518b20; no voters: bd146d3a022940a68274ddac0993c2e3
I20260619 10:17:33.666277 8201 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4bac10aeb06347f2bfadf295a3417d44 [term 1 FOLLOWER]: Starting leader election (detected failure of leader bd146d3a022940a68274ddac0993c2e3)
I20260619 10:17:33.666301 8201 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4bac10aeb06347f2bfadf295a3417d44 [term 1 FOLLOWER]: Advancing to term 2
I20260619 10:17:33.666347 8200 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cd27214b90874d35b38271d228518b20 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260619 10:17:33.666385 8200 raft_consensus.cc:482] T 00000000000000000000000000000000 P cd27214b90874d35b38271d228518b20 [term 1 FOLLOWER]: Starting leader election (detected failure of leader bd146d3a022940a68274ddac0993c2e3)
I20260619 10:17:33.666409 8200 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cd27214b90874d35b38271d228518b20 [term 1 FOLLOWER]: Advancing to term 2
I20260619 10:17:33.667160 8201 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4bac10aeb06347f2bfadf295a3417d44 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "bd146d3a022940a68274ddac0993c2e3" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 38663 } } peers { permanent_uuid: "cd27214b90874d35b38271d228518b20" member_type: VOTER last_known_addr { host: "127.0.206.125" port: 40043 } attrs { promote: false } } peers { permanent_uuid: "4bac10aeb06347f2bfadf295a3417d44" member_type: VOTER last_known_addr { host: "127.0.206.124" port: 42647 } attrs { promote: false } }
I20260619 10:17:33.667160 8200 raft_consensus.cc:504] T 00000000000000000000000000000000 P cd27214b90874d35b38271d228518b20 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "bd146d3a022940a68274ddac0993c2e3" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 38663 } } peers { permanent_uuid: "cd27214b90874d35b38271d228518b20" member_type: VOTER last_known_addr { host: "127.0.206.125" port: 40043 } attrs { promote: false } } peers { permanent_uuid: "4bac10aeb06347f2bfadf295a3417d44" member_type: VOTER last_known_addr { host: "127.0.206.124" port: 42647 } attrs { promote: false } }
I20260619 10:17:33.667274 8201 leader_election.cc:288] T 00000000000000000000000000000000 P 4bac10aeb06347f2bfadf295a3417d44 [CANDIDATE]: Term 2 election: Requested vote from peers bd146d3a022940a68274ddac0993c2e3 (127.0.206.126:38663), cd27214b90874d35b38271d228518b20 (127.0.206.125:40043)
I20260619 10:17:33.667286 8200 leader_election.cc:288] T 00000000000000000000000000000000 P cd27214b90874d35b38271d228518b20 [CANDIDATE]: Term 2 election: Requested vote from peers bd146d3a022940a68274ddac0993c2e3 (127.0.206.126:38663), 4bac10aeb06347f2bfadf295a3417d44 (127.0.206.124:42647)
I20260619 10:17:33.667428 8203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:33.667348 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:38663 (local address 127.0.0.1:37990)
0619 10:17:33.667381 (+ 33us) negotiation.cc:107] Waiting for socket to connect
0619 10:17:33.667407 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.126:38663: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":20}
I20260619 10:17:33.667480 7693 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "cd27214b90874d35b38271d228518b20" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "4bac10aeb06347f2bfadf295a3417d44"
I20260619 10:17:33.667520 7507 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "4bac10aeb06347f2bfadf295a3417d44" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "cd27214b90874d35b38271d228518b20"
I20260619 10:17:33.667558 7693 raft_consensus.cc:2379] T 00000000000000000000000000000000 P 4bac10aeb06347f2bfadf295a3417d44 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate cd27214b90874d35b38271d228518b20 in current term 2: Already voted for candidate 4bac10aeb06347f2bfadf295a3417d44 in this term.
I20260619 10:17:33.667586 7507 raft_consensus.cc:2379] T 00000000000000000000000000000000 P cd27214b90874d35b38271d228518b20 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 4bac10aeb06347f2bfadf295a3417d44 in current term 2: Already voted for candidate cd27214b90874d35b38271d228518b20 in this term.
I20260619 10:17:33.667707 8203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:33.667614 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:38663 (local address 127.0.0.1:38012)
0619 10:17:33.667661 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0619 10:17:33.667687 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.126:38663: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260619 10:17:33.667788 7483 leader_election.cc:334] T 00000000000000000000000000000000 P cd27214b90874d35b38271d228518b20 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer bd146d3a022940a68274ddac0993c2e3 (127.0.206.126:38663): Network error: Client connection negotiation failed: client connection to 127.0.206.126:38663: connect: Connection refused (error 111)
I20260619 10:17:33.667822 8204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:33.667403 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:38663 (local address 127.0.0.1:37998)
0619 10:17:33.667776 (+ 373us) negotiation.cc:107] Waiting for socket to connect
0619 10:17:33.667801 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.126:38663: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":361}
I20260619 10:17:33.667846 7483 leader_election.cc:302] T 00000000000000000000000000000000 P cd27214b90874d35b38271d228518b20 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd27214b90874d35b38271d228518b20; no voters: 4bac10aeb06347f2bfadf295a3417d44, bd146d3a022940a68274ddac0993c2e3
I20260619 10:17:33.667939 8200 raft_consensus.cc:2726] T 00000000000000000000000000000000 P cd27214b90874d35b38271d228518b20 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260619 10:17:33.668081 8204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:33.667997 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:38663 (local address 127.0.0.1:38024)
0619 10:17:33.668028 (+ 31us) negotiation.cc:107] Waiting for socket to connect
0619 10:17:33.668057 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.126:38663: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":17}
W20260619 10:17:33.668167 7669 leader_election.cc:334] T 00000000000000000000000000000000 P 4bac10aeb06347f2bfadf295a3417d44 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer bd146d3a022940a68274ddac0993c2e3 (127.0.206.126:38663): Network error: Client connection negotiation failed: client connection to 127.0.206.126:38663: connect: Connection refused (error 111)
I20260619 10:17:33.668239 7669 leader_election.cc:302] T 00000000000000000000000000000000 P 4bac10aeb06347f2bfadf295a3417d44 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4bac10aeb06347f2bfadf295a3417d44; no voters: bd146d3a022940a68274ddac0993c2e3, cd27214b90874d35b38271d228518b20
I20260619 10:17:33.668311 8201 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 4bac10aeb06347f2bfadf295a3417d44 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260619 10:17:34.159672 8210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:34.159460 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:38663 (local address 127.0.206.67:46811)
0619 10:17:34.159575 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0619 10:17:34.159616 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.126:38663: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":98,"thread_start_us":71,"threads_started":1}
I20260619 10:17:34.159670 8211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:34.159443 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:38663 (local address 127.0.206.65:52869)
0619 10:17:34.159576 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0619 10:17:34.159616 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.126:38663: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":89,"threads_started":1}
I20260619 10:17:34.160355 8212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:34.160195 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:38663 (local address 127.0.206.66:33951)
0619 10:17:34.160292 (+ 97us) negotiation.cc:107] Waiting for socket to connect
0619 10:17:34.160323 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.126:38663: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":84,"thread_start_us":51,"threads_started":1}
I20260619 10:17:35.160396 8213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:35.160173 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:38663 (local address 127.0.206.65:45125)
0619 10:17:35.160304 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0619 10:17:35.160344 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.126:38663: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":58,"threads_started":1}
I20260619 10:17:35.160619 8214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:35.160198 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:38663 (local address 127.0.206.67:33945)
0619 10:17:35.160519 (+ 321us) negotiation.cc:107] Waiting for socket to connect
0619 10:17:35.160576 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.126:38663: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":304,"thread_start_us":52,"threads_started":1}
I20260619 10:17:35.160925 8215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:35.160746 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:38663 (local address 127.0.206.66:44545)
0619 10:17:35.160847 (+ 101us) negotiation.cc:107] Waiting for socket to connect
0619 10:17:35.160881 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.126:38663: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":85,"thread_start_us":63,"threads_started":1}
I20260619 10:17:35.176626 8216 raft_consensus.cc:482] T 00000000000000000000000000000000 P cd27214b90874d35b38271d228518b20 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 10:17:35.176693 8216 raft_consensus.cc:504] T 00000000000000000000000000000000 P cd27214b90874d35b38271d228518b20 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "bd146d3a022940a68274ddac0993c2e3" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 38663 } } peers { permanent_uuid: "cd27214b90874d35b38271d228518b20" member_type: VOTER last_known_addr { host: "127.0.206.125" port: 40043 } attrs { promote: false } } peers { permanent_uuid: "4bac10aeb06347f2bfadf295a3417d44" member_type: VOTER last_known_addr { host: "127.0.206.124" port: 42647 } attrs { promote: false } }
I20260619 10:17:35.176858 8216 leader_election.cc:288] T 00000000000000000000000000000000 P cd27214b90874d35b38271d228518b20 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers bd146d3a022940a68274ddac0993c2e3 (127.0.206.126:38663), 4bac10aeb06347f2bfadf295a3417d44 (127.0.206.124:42647)
I20260619 10:17:35.177111 8217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:35.176953 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:38663 (local address 127.0.0.1:38222)
0619 10:17:35.177039 (+ 86us) negotiation.cc:107] Waiting for socket to connect
0619 10:17:35.177070 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.126:38663: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71,"thread_start_us":53,"threads_started":1}
I20260619 10:17:35.177137 7693 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "cd27214b90874d35b38271d228518b20" candidate_term: 3 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "4bac10aeb06347f2bfadf295a3417d44" is_pre_election: true
I20260619 10:17:35.177212 7693 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 4bac10aeb06347f2bfadf295a3417d44 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd27214b90874d35b38271d228518b20 in term 2.
I20260619 10:17:35.177369 7483 leader_election.cc:302] T 00000000000000000000000000000000 P cd27214b90874d35b38271d228518b20 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4bac10aeb06347f2bfadf295a3417d44, cd27214b90874d35b38271d228518b20; no voters:
I20260619 10:17:35.177486 8216 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cd27214b90874d35b38271d228518b20 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260619 10:17:35.177524 8216 raft_consensus.cc:482] T 00000000000000000000000000000000 P cd27214b90874d35b38271d228518b20 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 10:17:35.177538 8216 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cd27214b90874d35b38271d228518b20 [term 2 FOLLOWER]: Advancing to term 3
I20260619 10:17:35.177646 8217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:35.177516 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:38663 (local address 127.0.0.1:38232)
0619 10:17:35.177572 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0619 10:17:35.177613 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.126:38663: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260619 10:17:35.177704 7483 leader_election.cc:334] T 00000000000000000000000000000000 P cd27214b90874d35b38271d228518b20 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer bd146d3a022940a68274ddac0993c2e3 (127.0.206.126:38663): Network error: Client connection negotiation failed: client connection to 127.0.206.126:38663: connect: Connection refused (error 111)
I20260619 10:17:35.178279 8216 raft_consensus.cc:504] T 00000000000000000000000000000000 P cd27214b90874d35b38271d228518b20 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "bd146d3a022940a68274ddac0993c2e3" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 38663 } } peers { permanent_uuid: "cd27214b90874d35b38271d228518b20" member_type: VOTER last_known_addr { host: "127.0.206.125" port: 40043 } attrs { promote: false } } peers { permanent_uuid: "4bac10aeb06347f2bfadf295a3417d44" member_type: VOTER last_known_addr { host: "127.0.206.124" port: 42647 } attrs { promote: false } }
I20260619 10:17:35.178397 8216 leader_election.cc:288] T 00000000000000000000000000000000 P cd27214b90874d35b38271d228518b20 [CANDIDATE]: Term 3 election: Requested vote from peers bd146d3a022940a68274ddac0993c2e3 (127.0.206.126:38663), 4bac10aeb06347f2bfadf295a3417d44 (127.0.206.124:42647)
I20260619 10:17:35.178584 8217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:35.178489 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:38663 (local address 127.0.0.1:38248)
0619 10:17:35.178522 (+ 33us) negotiation.cc:107] Waiting for socket to connect
0619 10:17:35.178563 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.126:38663: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":20}
I20260619 10:17:35.178656 7693 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "cd27214b90874d35b38271d228518b20" candidate_term: 3 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "4bac10aeb06347f2bfadf295a3417d44"
I20260619 10:17:35.178727 7693 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4bac10aeb06347f2bfadf295a3417d44 [term 2 FOLLOWER]: Advancing to term 3
I20260619 10:17:35.178848 8217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:35.178785 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:38663 (local address 127.0.0.1:38252)
0619 10:17:35.178809 (+ 24us) negotiation.cc:107] Waiting for socket to connect
0619 10:17:35.178831 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.126:38663: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":12}
W20260619 10:17:35.178920 7483 leader_election.cc:334] T 00000000000000000000000000000000 P cd27214b90874d35b38271d228518b20 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer bd146d3a022940a68274ddac0993c2e3 (127.0.206.126:38663): Network error: Client connection negotiation failed: client connection to 127.0.206.126:38663: connect: Connection refused (error 111)
I20260619 10:17:35.179385 7693 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 4bac10aeb06347f2bfadf295a3417d44 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd27214b90874d35b38271d228518b20 in term 3.
I20260619 10:17:35.179576 7483 leader_election.cc:302] T 00000000000000000000000000000000 P cd27214b90874d35b38271d228518b20 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4bac10aeb06347f2bfadf295a3417d44, cd27214b90874d35b38271d228518b20; no voters: bd146d3a022940a68274ddac0993c2e3
I20260619 10:17:35.179683 8216 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cd27214b90874d35b38271d228518b20 [term 3 FOLLOWER]: Leader election won for term 3
I20260619 10:17:35.179795 8216 raft_consensus.cc:686] T 00000000000000000000000000000000 P cd27214b90874d35b38271d228518b20 [term 3 LEADER]: Becoming Leader. State: Replica: cd27214b90874d35b38271d228518b20, State: Running, Role: LEADER
I20260619 10:17:35.179872 8216 consensus_queue.cc:237] T 00000000000000000000000000000000 P cd27214b90874d35b38271d228518b20 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "bd146d3a022940a68274ddac0993c2e3" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 38663 } } peers { permanent_uuid: "cd27214b90874d35b38271d228518b20" member_type: VOTER last_known_addr { host: "127.0.206.125" port: 40043 } attrs { promote: false } } peers { permanent_uuid: "4bac10aeb06347f2bfadf295a3417d44" member_type: VOTER last_known_addr { host: "127.0.206.124" port: 42647 } attrs { promote: false } }
I20260619 10:17:35.180204 8219 sys_catalog.cc:455] T 00000000000000000000000000000000 P cd27214b90874d35b38271d228518b20 [sys.catalog]: SysCatalogTable state changed. Reason: New leader cd27214b90874d35b38271d228518b20. Latest consensus state: current_term: 3 leader_uuid: "cd27214b90874d35b38271d228518b20" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "bd146d3a022940a68274ddac0993c2e3" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 38663 } } peers { permanent_uuid: "cd27214b90874d35b38271d228518b20" member_type: VOTER last_known_addr { host: "127.0.206.125" port: 40043 } attrs { promote: false } } peers { permanent_uuid: "4bac10aeb06347f2bfadf295a3417d44" member_type: VOTER last_known_addr { host: "127.0.206.124" port: 42647 } attrs { promote: false } } }
I20260619 10:17:35.180298 8219 sys_catalog.cc:458] T 00000000000000000000000000000000 P cd27214b90874d35b38271d228518b20 [sys.catalog]: This master's current role is: LEADER
I20260619 10:17:35.180467 8221 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 10:17:35.180824 8221 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 10:17:35.180884 8221 catalog_manager.cc:1259] Loaded cluster ID: a825fc644b564a9897f7dc902ab7b888
I20260619 10:17:35.180907 8221 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 10:17:35.181051 8221 catalog_manager.cc:1504] Loading token signing keys...
I20260619 10:17:35.181113 8221 catalog_manager.cc:5879] T 00000000000000000000000000000000 P cd27214b90874d35b38271d228518b20: Loaded TSK: 0
I20260619 10:17:35.181205 8221 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 10:17:35.257017 7496 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48788:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 10:17:35.258750 7496 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 10:17:35.259765 8217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:35.259643 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:38663 (local address 127.0.0.1:38272)
0619 10:17:35.259705 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0619 10:17:35.259739 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.126:38663: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
I20260619 10:17:35.259869 7693 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 4bac10aeb06347f2bfadf295a3417d44 [term 3 FOLLOWER]: Refusing update from remote peer cd27214b90874d35b38271d228518b20: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 3 index: 14. (index mismatch)
I20260619 10:17:35.260074 8217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:35.259975 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:38663 (local address 127.0.0.1:38278)
0619 10:17:35.260022 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0619 10:17:35.260051 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.126:38663: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
I20260619 10:17:35.260116 8216 consensus_queue.cc:1035] T 00000000000000000000000000000000 P cd27214b90874d35b38271d228518b20 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4bac10aeb06347f2bfadf295a3417d44" member_type: VOTER last_known_addr { host: "127.0.206.124" port: 42647 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
W20260619 10:17:35.260236 7483 consensus_peers.cc:487] T 00000000000000000000000000000000 P cd27214b90874d35b38271d228518b20 -> Peer bd146d3a022940a68274ddac0993c2e3 (127.0.206.126:38663): Couldn't send request to peer bd146d3a022940a68274ddac0993c2e3. Status: Network error: Client connection negotiation failed: client connection to 127.0.206.126:38663: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260619 10:17:35.260704 8223 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4bac10aeb06347f2bfadf295a3417d44 [sys.catalog]: SysCatalogTable state changed. Reason: New leader cd27214b90874d35b38271d228518b20. Latest consensus state: current_term: 3 leader_uuid: "cd27214b90874d35b38271d228518b20" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "bd146d3a022940a68274ddac0993c2e3" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 38663 } } peers { permanent_uuid: "cd27214b90874d35b38271d228518b20" member_type: VOTER last_known_addr { host: "127.0.206.125" port: 40043 } attrs { promote: false } } peers { permanent_uuid: "4bac10aeb06347f2bfadf295a3417d44" member_type: VOTER last_known_addr { host: "127.0.206.124" port: 42647 } attrs { promote: false } } }
I20260619 10:17:35.260808 8223 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4bac10aeb06347f2bfadf295a3417d44 [sys.catalog]: This master's current role is: FOLLOWER
I20260619 10:17:35.261075 8222 mvcc.cc:205] Tried to move back new op lower bound from 7298515989541937152 to 7298515989217308672. Current Snapshot: MvccSnapshot[applied={T|T < 7298515989541937152}]
I20260619 10:17:35.261169 8219 sys_catalog.cc:455] T 00000000000000000000000000000000 P cd27214b90874d35b38271d228518b20 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 3 leader_uuid: "cd27214b90874d35b38271d228518b20" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "bd146d3a022940a68274ddac0993c2e3" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 38663 } } peers { permanent_uuid: "cd27214b90874d35b38271d228518b20" member_type: VOTER last_known_addr { host: "127.0.206.125" port: 40043 } attrs { promote: false } } peers { permanent_uuid: "4bac10aeb06347f2bfadf295a3417d44" member_type: VOTER last_known_addr { host: "127.0.206.124" port: 42647 } attrs { promote: false } } }
I20260619 10:17:35.261260 8219 sys_catalog.cc:458] T 00000000000000000000000000000000 P cd27214b90874d35b38271d228518b20 [sys.catalog]: This master's current role is: LEADER
I20260619 10:17:35.261524 8223 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4bac10aeb06347f2bfadf295a3417d44 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 3 leader_uuid: "cd27214b90874d35b38271d228518b20" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "bd146d3a022940a68274ddac0993c2e3" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 38663 } } peers { permanent_uuid: "cd27214b90874d35b38271d228518b20" member_type: VOTER last_known_addr { host: "127.0.206.125" port: 40043 } attrs { promote: false } } peers { permanent_uuid: "4bac10aeb06347f2bfadf295a3417d44" member_type: VOTER last_known_addr { host: "127.0.206.124" port: 42647 } attrs { promote: false } } }
I20260619 10:17:35.261606 8223 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4bac10aeb06347f2bfadf295a3417d44 [sys.catalog]: This master's current role is: FOLLOWER
I20260619 10:17:35.266443 8217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:35.262940 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.65:33905 (local address 127.0.0.1:38740)
0619 10:17:35.262982 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0619 10:17:35.262992 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 10:17:35.263037 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 10:17:35.263553 (+ 516us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 10:17:35.263554 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 10:17:35.263555 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 10:17:35.263604 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:35.263605 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:35.264555 (+ 950us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:35.264556 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:35.265493 (+ 937us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:35.265495 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:35.266196 (+ 701us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:35.266197 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:35.266258 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:35.266262 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 10:17:35.266322 (+ 60us) client_negotiation.cc:240] Negotiation successful
0619 10:17:35.266362 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":26,"mutex_wait_us":34}
I20260619 10:17:35.266571 8232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:35.263233 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.66:37571 (local address 127.0.0.1:59094)
0619 10:17:35.263635 (+ 402us) negotiation.cc:107] Waiting for socket to connect
0619 10:17:35.263645 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 10:17:35.263678 (+ 33us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 10:17:35.263809 (+ 131us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 10:17:35.263809 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 10:17:35.263811 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 10:17:35.263843 (+ 32us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:35.263844 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:35.264654 (+ 810us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:35.264655 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:35.265756 (+ 1101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:35.265758 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:35.266367 (+ 609us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:35.266368 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:35.266423 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:35.266426 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 10:17:35.266474 (+ 48us) client_negotiation.cc:240] Negotiation successful
0619 10:17:35.266514 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":386,"thread_start_us":86,"threads_started":1}
I20260619 10:17:35.266752 8229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:35.263006 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.67:43923 (local address 127.0.0.1:35398)
0619 10:17:35.263133 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0619 10:17:35.263144 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 10:17:35.263186 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 10:17:35.263499 (+ 313us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 10:17:35.263500 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 10:17:35.263503 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 10:17:35.263557 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:35.263559 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:35.264295 (+ 736us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:35.264296 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:35.265376 (+ 1080us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:35.265378 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:35.266496 (+ 1118us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:35.266497 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:35.266571 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:35.266575 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 10:17:35.266652 (+ 77us) client_negotiation.cc:240] Negotiation successful
0619 10:17:35.266691 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":73,"threads_started":1}
I20260619 10:17:35.266853 8230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:35.263149 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38740 (local address 127.0.206.65:33905)
0619 10:17:35.263433 (+ 284us) server_negotiation.cc:206] Beginning negotiation
0619 10:17:35.263435 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 10:17:35.263443 (+ 8us) server_negotiation.cc:407] Connection header received
0619 10:17:35.263469 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:17:35.263469 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:17:35.263480 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:17:35.263550 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 10:17:35.263694 (+ 144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:35.264467 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:35.265567 (+ 1100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:35.266123 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:35.266189 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:35.266201 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0619 10:17:35.266341 (+ 140us) server_negotiation.cc:299] Negotiation successful
0619 10:17:35.266384 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":262,"thread_start_us":57,"threads_started":1}
I20260619 10:17:35.266883 8233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:35.263433 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59094 (local address 127.0.206.66:37571)
0619 10:17:35.263693 (+ 260us) server_negotiation.cc:206] Beginning negotiation
0619 10:17:35.263695 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 10:17:35.263703 (+ 8us) server_negotiation.cc:407] Connection header received
0619 10:17:35.263728 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:17:35.263729 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:17:35.263739 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:17:35.263799 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 10:17:35.263896 (+ 97us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:35.264584 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:35.265833 (+ 1249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:35.266258 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:35.266328 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:35.266339 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0619 10:17:35.266699 (+ 360us) server_negotiation.cc:299] Negotiation successful
0619 10:17:35.266749 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":239,"thread_start_us":48,"threads_started":1}
I20260619 10:17:35.267073 7796 tablet_service.cc:1467] Processing CreateTablet for tablet a455abdb2ed24339a4a5515c6a4e5a49 (DEFAULT_TABLE table=test-table [id=1fef71b50c00404499f2d310d6ec5d11]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 10:17:35.267133 8231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:35.263185 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35398 (local address 127.0.206.67:43923)
0619 10:17:35.263272 (+ 87us) server_negotiation.cc:206] Beginning negotiation
0619 10:17:35.263274 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 10:17:35.263282 (+ 8us) server_negotiation.cc:407] Connection header received
0619 10:17:35.263313 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:17:35.263314 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:17:35.263322 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:17:35.263399 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 10:17:35.263617 (+ 218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:35.264223 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:35.265855 (+ 1632us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:35.266419 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:35.266965 (+ 546us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:35.266979 (+ 14us) server_negotiation.cc:1035] Waiting for connection context
0619 10:17:35.267018 (+ 39us) server_negotiation.cc:299] Negotiation successful
0619 10:17:35.267056 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68,"thread_start_us":44,"threads_started":1}
I20260619 10:17:35.267163 7933 tablet_service.cc:1467] Processing CreateTablet for tablet a455abdb2ed24339a4a5515c6a4e5a49 (DEFAULT_TABLE table=test-table [id=1fef71b50c00404499f2d310d6ec5d11]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 10:17:35.267203 7796 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a455abdb2ed24339a4a5515c6a4e5a49. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 10:17:35.267275 7933 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a455abdb2ed24339a4a5515c6a4e5a49. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 10:17:35.267297 8069 tablet_service.cc:1467] Processing CreateTablet for tablet a455abdb2ed24339a4a5515c6a4e5a49 (DEFAULT_TABLE table=test-table [id=1fef71b50c00404499f2d310d6ec5d11]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 10:17:35.267401 8069 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a455abdb2ed24339a4a5515c6a4e5a49. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 10:17:35.268401 8234 tablet_bootstrap.cc:492] T a455abdb2ed24339a4a5515c6a4e5a49 P 04a062435b8049fa92d4f33097d4e8a2: Bootstrap starting.
I20260619 10:17:35.268774 8236 tablet_bootstrap.cc:492] T a455abdb2ed24339a4a5515c6a4e5a49 P 9c6d725a83a94c8da04ad1daed4d3816: Bootstrap starting.
I20260619 10:17:35.268867 8234 tablet_bootstrap.cc:654] T a455abdb2ed24339a4a5515c6a4e5a49 P 04a062435b8049fa92d4f33097d4e8a2: Neither blocks nor log segments found. Creating new log.
I20260619 10:17:35.269130 8236 tablet_bootstrap.cc:654] T a455abdb2ed24339a4a5515c6a4e5a49 P 9c6d725a83a94c8da04ad1daed4d3816: Neither blocks nor log segments found. Creating new log.
I20260619 10:17:35.269433 8235 tablet_bootstrap.cc:492] T a455abdb2ed24339a4a5515c6a4e5a49 P 67adfa875ae04601b7b8fceb7610166c: Bootstrap starting.
I20260619 10:17:35.269440 8234 tablet_bootstrap.cc:492] T a455abdb2ed24339a4a5515c6a4e5a49 P 04a062435b8049fa92d4f33097d4e8a2: No bootstrap required, opened a new log
I20260619 10:17:35.269497 8234 ts_tablet_manager.cc:1397] T a455abdb2ed24339a4a5515c6a4e5a49 P 04a062435b8049fa92d4f33097d4e8a2: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260619 10:17:35.269681 8236 tablet_bootstrap.cc:492] T a455abdb2ed24339a4a5515c6a4e5a49 P 9c6d725a83a94c8da04ad1daed4d3816: No bootstrap required, opened a new log
I20260619 10:17:35.269641 8234 raft_consensus.cc:348] T a455abdb2ed24339a4a5515c6a4e5a49 P 04a062435b8049fa92d4f33097d4e8a2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67adfa875ae04601b7b8fceb7610166c" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 33905 } } peers { permanent_uuid: "9c6d725a83a94c8da04ad1daed4d3816" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 43923 } } peers { permanent_uuid: "04a062435b8049fa92d4f33097d4e8a2" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 37571 } }
I20260619 10:17:35.269737 8234 raft_consensus.cc:374] T a455abdb2ed24339a4a5515c6a4e5a49 P 04a062435b8049fa92d4f33097d4e8a2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 10:17:35.269748 8236 ts_tablet_manager.cc:1397] T a455abdb2ed24339a4a5515c6a4e5a49 P 9c6d725a83a94c8da04ad1daed4d3816: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260619 10:17:35.269762 8234 raft_consensus.cc:729] T a455abdb2ed24339a4a5515c6a4e5a49 P 04a062435b8049fa92d4f33097d4e8a2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 04a062435b8049fa92d4f33097d4e8a2, State: Initialized, Role: FOLLOWER
I20260619 10:17:35.269801 8235 tablet_bootstrap.cc:654] T a455abdb2ed24339a4a5515c6a4e5a49 P 67adfa875ae04601b7b8fceb7610166c: Neither blocks nor log segments found. Creating new log.
I20260619 10:17:35.269831 8234 consensus_queue.cc:260] T a455abdb2ed24339a4a5515c6a4e5a49 P 04a062435b8049fa92d4f33097d4e8a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67adfa875ae04601b7b8fceb7610166c" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 33905 } } peers { permanent_uuid: "9c6d725a83a94c8da04ad1daed4d3816" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 43923 } } peers { permanent_uuid: "04a062435b8049fa92d4f33097d4e8a2" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 37571 } }
I20260619 10:17:35.269862 8236 raft_consensus.cc:348] T a455abdb2ed24339a4a5515c6a4e5a49 P 9c6d725a83a94c8da04ad1daed4d3816 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67adfa875ae04601b7b8fceb7610166c" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 33905 } } peers { permanent_uuid: "9c6d725a83a94c8da04ad1daed4d3816" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 43923 } } peers { permanent_uuid: "04a062435b8049fa92d4f33097d4e8a2" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 37571 } }
I20260619 10:17:35.269946 8236 raft_consensus.cc:374] T a455abdb2ed24339a4a5515c6a4e5a49 P 9c6d725a83a94c8da04ad1daed4d3816 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 10:17:35.269974 8236 raft_consensus.cc:729] T a455abdb2ed24339a4a5515c6a4e5a49 P 9c6d725a83a94c8da04ad1daed4d3816 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c6d725a83a94c8da04ad1daed4d3816, State: Initialized, Role: FOLLOWER
I20260619 10:17:35.269985 8234 ts_tablet_manager.cc:1428] T a455abdb2ed24339a4a5515c6a4e5a49 P 04a062435b8049fa92d4f33097d4e8a2: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260619 10:17:35.270033 8236 consensus_queue.cc:260] T a455abdb2ed24339a4a5515c6a4e5a49 P 9c6d725a83a94c8da04ad1daed4d3816 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67adfa875ae04601b7b8fceb7610166c" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 33905 } } peers { permanent_uuid: "9c6d725a83a94c8da04ad1daed4d3816" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 43923 } } peers { permanent_uuid: "04a062435b8049fa92d4f33097d4e8a2" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 37571 } }
I20260619 10:17:35.270195 8236 ts_tablet_manager.cc:1428] T a455abdb2ed24339a4a5515c6a4e5a49 P 9c6d725a83a94c8da04ad1daed4d3816: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260619 10:17:35.270305 8215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:35.270197 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:38663 (local address 127.0.206.66:56937)
0619 10:17:35.270239 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0619 10:17:35.270273 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.126:38663: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":22}
I20260619 10:17:35.270382 8235 tablet_bootstrap.cc:492] T a455abdb2ed24339a4a5515c6a4e5a49 P 67adfa875ae04601b7b8fceb7610166c: No bootstrap required, opened a new log
I20260619 10:17:35.270426 8235 ts_tablet_manager.cc:1397] T a455abdb2ed24339a4a5515c6a4e5a49 P 67adfa875ae04601b7b8fceb7610166c: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260619 10:17:35.270551 8235 raft_consensus.cc:348] T a455abdb2ed24339a4a5515c6a4e5a49 P 67adfa875ae04601b7b8fceb7610166c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67adfa875ae04601b7b8fceb7610166c" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 33905 } } peers { permanent_uuid: "9c6d725a83a94c8da04ad1daed4d3816" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 43923 } } peers { permanent_uuid: "04a062435b8049fa92d4f33097d4e8a2" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 37571 } }
I20260619 10:17:35.270650 8235 raft_consensus.cc:374] T a455abdb2ed24339a4a5515c6a4e5a49 P 67adfa875ae04601b7b8fceb7610166c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 10:17:35.270687 8235 raft_consensus.cc:729] T a455abdb2ed24339a4a5515c6a4e5a49 P 67adfa875ae04601b7b8fceb7610166c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 67adfa875ae04601b7b8fceb7610166c, State: Initialized, Role: FOLLOWER
I20260619 10:17:35.270730 8235 consensus_queue.cc:260] T a455abdb2ed24339a4a5515c6a4e5a49 P 67adfa875ae04601b7b8fceb7610166c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67adfa875ae04601b7b8fceb7610166c" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 33905 } } peers { permanent_uuid: "9c6d725a83a94c8da04ad1daed4d3816" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 43923 } } peers { permanent_uuid: "04a062435b8049fa92d4f33097d4e8a2" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 37571 } }
I20260619 10:17:35.270910 8235 ts_tablet_manager.cc:1428] T a455abdb2ed24339a4a5515c6a4e5a49 P 67adfa875ae04601b7b8fceb7610166c: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260619 10:17:35.271031 8215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:35.270934 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:38663 (local address 127.0.206.66:45071)
0619 10:17:35.270977 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0619 10:17:35.271001 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.126:38663: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
I20260619 10:17:35.271130 8001 heartbeater.cc:503] Master 127.0.206.125:40043 was elected leader, sending a full tablet report...
I20260619 10:17:35.271363 8213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:35.271254 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:38663 (local address 127.0.206.65:41651)
0619 10:17:35.271295 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0619 10:17:35.271330 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.126:38663: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":22}
I20260619 10:17:35.271458 8214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:35.271082 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:38663 (local address 127.0.206.67:60919)
0619 10:17:35.271396 (+ 314us) negotiation.cc:107] Waiting for socket to connect
0619 10:17:35.271426 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.126:38663: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":298}
I20260619 10:17:35.271540 8137 heartbeater.cc:503] Master 127.0.206.125:40043 was elected leader, sending a full tablet report...
I20260619 10:17:35.272149 8214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:35.272039 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:38663 (local address 127.0.206.67:33059)
0619 10:17:35.272090 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0619 10:17:35.272122 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.126:38663: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":18}
I20260619 10:17:35.272413 7864 heartbeater.cc:503] Master 127.0.206.125:40043 was elected leader, sending a full tablet report...
I20260619 10:17:35.272640 8213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:35.272541 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:38663 (local address 127.0.206.65:46309)
0619 10:17:35.272589 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0619 10:17:35.272614 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.126:38663: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
I20260619 10:17:35.329356 8242 raft_consensus.cc:482] T a455abdb2ed24339a4a5515c6a4e5a49 P 67adfa875ae04601b7b8fceb7610166c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 10:17:35.329465 8242 raft_consensus.cc:504] T a455abdb2ed24339a4a5515c6a4e5a49 P 67adfa875ae04601b7b8fceb7610166c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67adfa875ae04601b7b8fceb7610166c" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 33905 } } peers { permanent_uuid: "9c6d725a83a94c8da04ad1daed4d3816" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 43923 } } peers { permanent_uuid: "04a062435b8049fa92d4f33097d4e8a2" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 37571 } }
I20260619 10:17:35.329828 8242 leader_election.cc:288] T a455abdb2ed24339a4a5515c6a4e5a49 P 67adfa875ae04601b7b8fceb7610166c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9c6d725a83a94c8da04ad1daed4d3816 (127.0.206.67:43923), 04a062435b8049fa92d4f33097d4e8a2 (127.0.206.66:37571)
I20260619 10:17:35.332633 8233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:35.330124 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.65:58475 (local address 127.0.206.66:37571)
0619 10:17:35.330190 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0619 10:17:35.330192 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 10:17:35.330200 (+ 8us) server_negotiation.cc:407] Connection header received
0619 10:17:35.330231 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:17:35.330232 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:17:35.330239 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:17:35.330301 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 10:17:35.330436 (+ 135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:35.331010 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:35.331955 (+ 945us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:35.332312 (+ 357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:35.332393 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:35.332402 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0619 10:17:35.332539 (+ 137us) server_negotiation.cc:299] Negotiation successful
0619 10:17:35.332577 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260619 10:17:35.332630 8213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:35.330004 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.66:37571 (local address 127.0.206.65:58475)
0619 10:17:35.330045 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0619 10:17:35.330057 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 10:17:35.330124 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 10:17:35.330303 (+ 179us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 10:17:35.330304 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 10:17:35.330307 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 10:17:35.330369 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:35.330370 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:35.331091 (+ 721us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:35.331092 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:35.331858 (+ 766us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:35.331860 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:35.332392 (+ 532us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:35.332392 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:35.332460 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:35.332463 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 10:17:35.332521 (+ 58us) client_negotiation.cc:240] Negotiation successful
0619 10:17:35.332558 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":19}
I20260619 10:17:35.332726 8231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:35.330082 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.65:38353 (local address 127.0.206.67:43923)
0619 10:17:35.330151 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0619 10:17:35.330154 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 10:17:35.330414 (+ 260us) server_negotiation.cc:407] Connection header received
0619 10:17:35.330485 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:17:35.330486 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:17:35.330494 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:17:35.330558 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 10:17:35.330690 (+ 132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:35.331197 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:35.332058 (+ 861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:35.332409 (+ 351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:35.332483 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:35.332492 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0619 10:17:35.332629 (+ 137us) server_negotiation.cc:299] Negotiation successful
0619 10:17:35.332664 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260619 10:17:35.332744 8243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:35.330004 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.67:43923 (local address 127.0.206.65:38353)
0619 10:17:35.330343 (+ 339us) negotiation.cc:107] Waiting for socket to connect
0619 10:17:35.330352 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0619 10:17:35.330422 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 10:17:35.330559 (+ 137us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 10:17:35.330559 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 10:17:35.330561 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 10:17:35.330626 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:35.330627 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:35.331277 (+ 650us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:35.331278 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:35.331976 (+ 698us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:35.331977 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:35.332512 (+ 535us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:35.332513 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:35.332564 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:35.332567 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 10:17:35.332624 (+ 57us) client_negotiation.cc:240] Negotiation successful
0619 10:17:35.332660 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":307,"mutex_wait_us":148,"thread_start_us":128,"threads_started":1}
I20260619 10:17:35.332908 7953 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a455abdb2ed24339a4a5515c6a4e5a49" candidate_uuid: "67adfa875ae04601b7b8fceb7610166c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "04a062435b8049fa92d4f33097d4e8a2" is_pre_election: true
I20260619 10:17:35.332971 8089 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a455abdb2ed24339a4a5515c6a4e5a49" candidate_uuid: "67adfa875ae04601b7b8fceb7610166c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9c6d725a83a94c8da04ad1daed4d3816" is_pre_election: true
I20260619 10:17:35.333047 8089 raft_consensus.cc:2454] T a455abdb2ed24339a4a5515c6a4e5a49 P 9c6d725a83a94c8da04ad1daed4d3816 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 67adfa875ae04601b7b8fceb7610166c in term 0.
I20260619 10:17:35.333051 7953 raft_consensus.cc:2454] T a455abdb2ed24339a4a5515c6a4e5a49 P 04a062435b8049fa92d4f33097d4e8a2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 67adfa875ae04601b7b8fceb7610166c in term 0.
I20260619 10:17:35.333199 7750 leader_election.cc:302] T a455abdb2ed24339a4a5515c6a4e5a49 P 67adfa875ae04601b7b8fceb7610166c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 67adfa875ae04601b7b8fceb7610166c, 9c6d725a83a94c8da04ad1daed4d3816; no voters:
I20260619 10:17:35.333344 8242 raft_consensus.cc:2781] T a455abdb2ed24339a4a5515c6a4e5a49 P 67adfa875ae04601b7b8fceb7610166c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 10:17:35.333403 8242 raft_consensus.cc:482] T a455abdb2ed24339a4a5515c6a4e5a49 P 67adfa875ae04601b7b8fceb7610166c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 10:17:35.333441 8242 raft_consensus.cc:3037] T a455abdb2ed24339a4a5515c6a4e5a49 P 67adfa875ae04601b7b8fceb7610166c [term 0 FOLLOWER]: Advancing to term 1
I20260619 10:17:35.334115 8242 raft_consensus.cc:504] T a455abdb2ed24339a4a5515c6a4e5a49 P 67adfa875ae04601b7b8fceb7610166c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67adfa875ae04601b7b8fceb7610166c" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 33905 } } peers { permanent_uuid: "9c6d725a83a94c8da04ad1daed4d3816" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 43923 } } peers { permanent_uuid: "04a062435b8049fa92d4f33097d4e8a2" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 37571 } }
I20260619 10:17:35.334317 8242 leader_election.cc:288] T a455abdb2ed24339a4a5515c6a4e5a49 P 67adfa875ae04601b7b8fceb7610166c [CANDIDATE]: Term 1 election: Requested vote from peers 9c6d725a83a94c8da04ad1daed4d3816 (127.0.206.67:43923), 04a062435b8049fa92d4f33097d4e8a2 (127.0.206.66:37571)
I20260619 10:17:35.334373 7953 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a455abdb2ed24339a4a5515c6a4e5a49" candidate_uuid: "67adfa875ae04601b7b8fceb7610166c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "04a062435b8049fa92d4f33097d4e8a2"
I20260619 10:17:35.334432 7953 raft_consensus.cc:3037] T a455abdb2ed24339a4a5515c6a4e5a49 P 04a062435b8049fa92d4f33097d4e8a2 [term 0 FOLLOWER]: Advancing to term 1
I20260619 10:17:35.334419 8089 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a455abdb2ed24339a4a5515c6a4e5a49" candidate_uuid: "67adfa875ae04601b7b8fceb7610166c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9c6d725a83a94c8da04ad1daed4d3816"
I20260619 10:17:35.334484 8089 raft_consensus.cc:3037] T a455abdb2ed24339a4a5515c6a4e5a49 P 9c6d725a83a94c8da04ad1daed4d3816 [term 0 FOLLOWER]: Advancing to term 1
I20260619 10:17:35.335152 7953 raft_consensus.cc:2454] T a455abdb2ed24339a4a5515c6a4e5a49 P 04a062435b8049fa92d4f33097d4e8a2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 67adfa875ae04601b7b8fceb7610166c in term 1.
I20260619 10:17:35.335178 8089 raft_consensus.cc:2454] T a455abdb2ed24339a4a5515c6a4e5a49 P 9c6d725a83a94c8da04ad1daed4d3816 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 67adfa875ae04601b7b8fceb7610166c in term 1.
I20260619 10:17:35.335283 7751 leader_election.cc:302] T a455abdb2ed24339a4a5515c6a4e5a49 P 67adfa875ae04601b7b8fceb7610166c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 04a062435b8049fa92d4f33097d4e8a2, 67adfa875ae04601b7b8fceb7610166c; no voters:
I20260619 10:17:35.335374 8242 raft_consensus.cc:2781] T a455abdb2ed24339a4a5515c6a4e5a49 P 67adfa875ae04601b7b8fceb7610166c [term 1 FOLLOWER]: Leader election won for term 1
I20260619 10:17:35.335523 8242 raft_consensus.cc:686] T a455abdb2ed24339a4a5515c6a4e5a49 P 67adfa875ae04601b7b8fceb7610166c [term 1 LEADER]: Becoming Leader. State: Replica: 67adfa875ae04601b7b8fceb7610166c, State: Running, Role: LEADER
I20260619 10:17:35.335700 8242 consensus_queue.cc:237] T a455abdb2ed24339a4a5515c6a4e5a49 P 67adfa875ae04601b7b8fceb7610166c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67adfa875ae04601b7b8fceb7610166c" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 33905 } } peers { permanent_uuid: "9c6d725a83a94c8da04ad1daed4d3816" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 43923 } } peers { permanent_uuid: "04a062435b8049fa92d4f33097d4e8a2" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 37571 } }
I20260619 10:17:35.335806 8243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:35.335701 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:38663 (local address 127.0.206.65:54743)
0619 10:17:35.335742 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0619 10:17:35.335778 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.126:38663: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
I20260619 10:17:35.336544 7495 catalog_manager.cc:5495] T a455abdb2ed24339a4a5515c6a4e5a49 P 67adfa875ae04601b7b8fceb7610166c reported cstate change: term changed from 0 to 1, leader changed from <none> to 67adfa875ae04601b7b8fceb7610166c (127.0.206.65). New cstate: current_term: 1 leader_uuid: "67adfa875ae04601b7b8fceb7610166c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "67adfa875ae04601b7b8fceb7610166c" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 33905 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9c6d725a83a94c8da04ad1daed4d3816" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 43923 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "04a062435b8049fa92d4f33097d4e8a2" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 37571 } health_report { overall_health: UNKNOWN } } }
I20260619 10:17:35.358790 8230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:35.356514 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38754 (local address 127.0.206.65:33905)
0619 10:17:35.356555 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0619 10:17:35.356557 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 10:17:35.356564 (+ 7us) server_negotiation.cc:407] Connection header received
0619 10:17:35.356638 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:17:35.356639 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:17:35.356656 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:17:35.356712 (+ 56us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 10:17:35.356814 (+ 102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:35.357319 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:35.358064 (+ 745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:35.358239 (+ 175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:35.358317 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:35.358421 (+ 104us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 10:17:35.358524 (+ 103us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 10:17:35.358577 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0619 10:17:35.358670 (+ 93us) server_negotiation.cc:299] Negotiation successful
0619 10:17:35.358694 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260619 10:17:35.359823 7953 raft_consensus.cc:1261] T a455abdb2ed24339a4a5515c6a4e5a49 P 04a062435b8049fa92d4f33097d4e8a2 [term 1 FOLLOWER]: Refusing update from remote peer 67adfa875ae04601b7b8fceb7610166c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 10:17:35.360064 8089 raft_consensus.cc:1261] T a455abdb2ed24339a4a5515c6a4e5a49 P 9c6d725a83a94c8da04ad1daed4d3816 [term 1 FOLLOWER]: Refusing update from remote peer 67adfa875ae04601b7b8fceb7610166c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 10:17:35.360183 8244 consensus_queue.cc:1035] T a455abdb2ed24339a4a5515c6a4e5a49 P 67adfa875ae04601b7b8fceb7610166c [LEADER]: Connected to new peer: Peer: permanent_uuid: "04a062435b8049fa92d4f33097d4e8a2" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 37571 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 10:17:35.360314 8242 consensus_queue.cc:1035] T a455abdb2ed24339a4a5515c6a4e5a49 P 67adfa875ae04601b7b8fceb7610166c [LEADER]: Connected to new peer: Peer: permanent_uuid: "9c6d725a83a94c8da04ad1daed4d3816" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 43923 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 10:17:35.361222 8215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:35.361097 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:38663 (local address 127.0.206.66:43257)
0619 10:17:35.361151 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0619 10:17:35.361187 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.126:38663: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
I20260619 10:17:35.361299 8214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:35.361167 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:38663 (local address 127.0.206.67:60073)
0619 10:17:35.361236 (+ 69us) negotiation.cc:107] Waiting for socket to connect
0619 10:17:35.361270 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.126:38663: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
I20260619 10:17:35.362157 8243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:35.362038 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:38663 (local address 127.0.206.65:41897)
0619 10:17:35.362101 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0619 10:17:35.362129 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.126:38663: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260619 10:17:35.362622 8215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:35.362478 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:38663 (local address 127.0.206.66:36765)
0619 10:17:35.362541 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0619 10:17:35.362572 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.126:38663: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
I20260619 10:17:35.363399 8243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:35.363317 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:38663 (local address 127.0.206.65:38905)
0619 10:17:35.363347 (+ 30us) negotiation.cc:107] Waiting for socket to connect
0619 10:17:35.363373 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.126:38663: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":16}
I20260619 10:17:35.364455 7496 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48788:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 10:17:35.364543 7496 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48788:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 10:17:35.364794 8214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:35.364679 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:38663 (local address 127.0.206.67:36227)
0619 10:17:35.364739 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0619 10:17:35.364770 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.126:38663: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
I20260619 10:17:35.365794 7496 catalog_manager.cc:5782] T 00000000000000000000000000000000 P cd27214b90874d35b38271d228518b20: Sending DeleteTablet for 3 replicas of tablet a455abdb2ed24339a4a5515c6a4e5a49
I20260619 10:17:35.366019 8068 tablet_service.cc:1514] Processing DeleteTablet for tablet a455abdb2ed24339a4a5515c6a4e5a49 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 10:17:35 UTC) from {username='oryx', principal='oryx/127.0.206.125@KRBTEST.COM'} at 127.0.0.1:35398
I20260619 10:17:35.366052 7796 tablet_service.cc:1514] Processing DeleteTablet for tablet a455abdb2ed24339a4a5515c6a4e5a49 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 10:17:35 UTC) from {username='oryx', principal='oryx/127.0.206.125@KRBTEST.COM'} at 127.0.0.1:38740
I20260619 10:17:35.366029 8069 tablet_service.cc:1514] Processing DeleteTablet for tablet a455abdb2ed24339a4a5515c6a4e5a49 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 10:17:35 UTC) from {username='oryx', principal='oryx/127.0.206.125@KRBTEST.COM'} at 127.0.0.1:35398
I20260619 10:17:35.366423 7933 tablet_service.cc:1514] Processing DeleteTablet for tablet a455abdb2ed24339a4a5515c6a4e5a49 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 10:17:35 UTC) from {username='oryx', principal='oryx/127.0.206.125@KRBTEST.COM'} at 127.0.0.1:59094
I20260619 10:17:35.366585 8259 tablet_replica.cc:330] T a455abdb2ed24339a4a5515c6a4e5a49 P 04a062435b8049fa92d4f33097d4e8a2: stopping tablet replica
I20260619 10:17:35.366684 825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTU9OP6/build/release/bin/kudu with pid 7645
I20260619 10:17:35.367482 8259 raft_consensus.cc:2229] T a455abdb2ed24339a4a5515c6a4e5a49 P 04a062435b8049fa92d4f33097d4e8a2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 10:17:35.367576 8259 raft_consensus.cc:2258] T a455abdb2ed24339a4a5515c6a4e5a49 P 04a062435b8049fa92d4f33097d4e8a2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 10:17:35.367885 8259 ts_tablet_manager.cc:1905] T a455abdb2ed24339a4a5515c6a4e5a49 P 04a062435b8049fa92d4f33097d4e8a2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 10:17:35.369649 8259 ts_tablet_manager.cc:1918] T a455abdb2ed24339a4a5515c6a4e5a49 P 04a062435b8049fa92d4f33097d4e8a2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 10:17:35.369709 8259 log.cc:1196] T a455abdb2ed24339a4a5515c6a4e5a49 P 04a062435b8049fa92d4f33097d4e8a2: Deleting WAL directory at /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781864224577058-825-0/minicluster-data/ts-1/wal/wals/a455abdb2ed24339a4a5515c6a4e5a49
I20260619 10:17:35.369949 8259 ts_tablet_manager.cc:1939] T a455abdb2ed24339a4a5515c6a4e5a49 P 04a062435b8049fa92d4f33097d4e8a2: Deleting consensus metadata
I20260619 10:17:35.370270 7483 catalog_manager.cc:4841] TS 04a062435b8049fa92d4f33097d4e8a2 (127.0.206.66:37571): tablet a455abdb2ed24339a4a5515c6a4e5a49 (table test-table [id=1fef71b50c00404499f2d310d6ec5d11]) successfully deleted
I20260619 10:17:35.372958 8229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:35.372855 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.65:33905 (local address 127.0.0.1:38758)
0619 10:17:35.372903 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0619 10:17:35.372927 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.65:33905: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
I20260619 10:17:35.373015 8257 tablet_replica.cc:330] T a455abdb2ed24339a4a5515c6a4e5a49 P 9c6d725a83a94c8da04ad1daed4d3816: stopping tablet replica
I20260619 10:17:35.373070 8257 raft_consensus.cc:2229] T a455abdb2ed24339a4a5515c6a4e5a49 P 9c6d725a83a94c8da04ad1daed4d3816 [term 1 FOLLOWER]: Raft consensus shutting down.
W20260619 10:17:35.373080 7484 catalog_manager.cc:4584] TS 67adfa875ae04601b7b8fceb7610166c (127.0.206.65:33905): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet a455abdb2ed24339a4a5515c6a4e5a49: Network error: Client connection negotiation failed: client connection to 127.0.206.65:33905: connect: Connection refused (error 111)
I20260619 10:17:35.373113 8257 raft_consensus.cc:2258] T a455abdb2ed24339a4a5515c6a4e5a49 P 9c6d725a83a94c8da04ad1daed4d3816 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 10:17:35.373312 825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTU9OP6/build/release/bin/kudu with pid 7870
I20260619 10:17:35.373350 8257 ts_tablet_manager.cc:1905] T a455abdb2ed24339a4a5515c6a4e5a49 P 9c6d725a83a94c8da04ad1daed4d3816: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 10:17:35.374732 8257 ts_tablet_manager.cc:1918] T a455abdb2ed24339a4a5515c6a4e5a49 P 9c6d725a83a94c8da04ad1daed4d3816: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 10:17:35.374800 8257 log.cc:1196] T a455abdb2ed24339a4a5515c6a4e5a49 P 9c6d725a83a94c8da04ad1daed4d3816: Deleting WAL directory at /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781864224577058-825-0/minicluster-data/ts-2/wal/wals/a455abdb2ed24339a4a5515c6a4e5a49
I20260619 10:17:35.375049 8257 ts_tablet_manager.cc:1939] T a455abdb2ed24339a4a5515c6a4e5a49 P 9c6d725a83a94c8da04ad1daed4d3816: Deleting consensus metadata
I20260619 10:17:35.375327 7482 catalog_manager.cc:4841] TS 9c6d725a83a94c8da04ad1daed4d3816 (127.0.206.67:43923): tablet a455abdb2ed24339a4a5515c6a4e5a49 (table test-table [id=1fef71b50c00404499f2d310d6ec5d11]) successfully deleted
W20260619 10:17:35.375558 7482 catalog_manager.cc:4808] TS 9c6d725a83a94c8da04ad1daed4d3816 (127.0.206.67:43923): delete failed for tablet a455abdb2ed24339a4a5515c6a4e5a49 because the tablet was not found. No further retry: Not found: Tablet not found: a455abdb2ed24339a4a5515c6a4e5a49
I20260619 10:17:35.379628 825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTU9OP6/build/release/bin/kudu with pid 8006
I20260619 10:17:35.383958 825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTU9OP6/build/release/bin/kudu with pid 7362
I20260619 10:17:35.388429 825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTU9OP6/build/release/bin/kudu with pid 7551
2026-06-19T10:17:35Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (6040 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781864224577058-825-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 10:17:35 dist-test-slave-6pt2 krb5kdc[8262](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 10:17:35 dist-test-slave-6pt2 krb5kdc[8262](info): set up 2 sockets
Jun 19 10:17:35 dist-test-slave-6pt2 krb5kdc[8262](info): commencing operation
krb5kdc: starting...
W20260619 10:17:36.415252 825 mini_kdc.cc:121] Time spent starting KDC: real 1.006s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781864224577058-825-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781864224577058-825-0/krb5kdc/test-user.keytab.
Jun 19 10:17:36 dist-test-slave-6pt2 krb5kdc[8262](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864256, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T10:17:36Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T10:17:36Z Disabled control of system clock
WARNING: no policy specified for oryx/127.0.206.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.0.206.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.0.206.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781864224577058-825-0/krb5kdc/oryx_127.0.206.126.keytab.
Entry for principal oryx/127.0.206.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781864224577058-825-0/krb5kdc/oryx_127.0.206.126.keytab.
I20260619 10:17:36.524659 825 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTU9OP6/build/release/bin/kudu
/tmp/dist-test-taskTU9OP6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781864224577058-825-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781864224577058-825-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781864224577058-825-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781864224577058-825-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.206.126:45657
--webserver_interface=127.0.206.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781864224577058-825-0/krb5kdc/oryx_127.0.206.126.keytab
--principal=oryx/127.0.206.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.206.84:41459
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.206.126:45657
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781864224577058-825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781864224577058-825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781864224577058-825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 10:17:36.602646 8275 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 10:17:36.602861 8275 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 10:17:36.602883 8275 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 10:17:36.604059 8275 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 10:17:36.604099 8275 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260619 10:17:36.604113 8275 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 10:17:36.604158 8275 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 10:17:36.604182 8275 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 10:17:36.604205 8275 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 10:17:36.606333 8275 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.206.84:41459
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781864224577058-825-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781864224577058-825-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.206.126:45657
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781864224577058-825-0/krb5kdc/oryx_127.0.206.126.keytab
--principal=oryx/127.0.206.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.206.126:45657
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781864224577058-825-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.206.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8275
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781864224577058-825-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 09:32:44 UTC on ccycloud.quasar-gbtldp.root.comops.site
build id 4455440
I20260619 10:17:36.606587 8275 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 10:17:36.606863 8275 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 10:17:36.612505 8281 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 10:17:36.612531 8283 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 10:17:36.612568 8280 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 10:17:36.612972 8275 server_base.cc:1034] running on GCE node
I20260619 10:17:36.613267 8275 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 10:17:36.613608 8275 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 10:17:36.614777 8275 hybrid_clock.cc:648] HybridClock initialized: now 1781864256614746 us; error 47 us; skew 500 ppm
Jun 19 10:17:36 dist-test-slave-6pt2 krb5kdc[8262](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864256, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.206.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 10:17:36.617403 8275 init.cc:377] Logged in from keytab as oryx/127.0.206.126@KRBTEST.COM (short username oryx)
I20260619 10:17:36.617838 8275 webserver.cc:460] Webserver started at http://127.0.206.126:33431/ using document root <none> and password file <none>
I20260619 10:17:36.618078 8275 fs_manager.cc:359] Metadata directory not provided
I20260619 10:17:36.618140 8275 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 10:17:36.618268 8275 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 10:17:36.619323 8275 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781864224577058-825-0/minicluster-data/master-0/data/instance:
uuid: "f99ceb70a6684009b753712e007c5de5"
format_stamp: "Formatted at 2026-06-19 10:17:36 on dist-test-slave-6pt2"
server_key: "3f591d9222d4682b13595efcfa75a239"
server_key_iv: "77c21ee9c624b0138435af2fbf7a226d"
server_key_version: "encryptionkey@0"
I20260619 10:17:36.619657 8275 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781864224577058-825-0/minicluster-data/master-0/wal/instance:
uuid: "f99ceb70a6684009b753712e007c5de5"
format_stamp: "Formatted at 2026-06-19 10:17:36 on dist-test-slave-6pt2"
server_key: "3f591d9222d4682b13595efcfa75a239"
server_key_iv: "77c21ee9c624b0138435af2fbf7a226d"
server_key_version: "encryptionkey@0"
I20260619 10:17:36.620854 8275 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260619 10:17:36.621533 8290 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 10:17:36.621693 8275 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260619 10:17:36.621783 8275 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781864224577058-825-0/minicluster-data/master-0/data,/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781864224577058-825-0/minicluster-data/master-0/wal
uuid: "f99ceb70a6684009b753712e007c5de5"
format_stamp: "Formatted at 2026-06-19 10:17:36 on dist-test-slave-6pt2"
server_key: "3f591d9222d4682b13595efcfa75a239"
server_key_iv: "77c21ee9c624b0138435af2fbf7a226d"
server_key_version: "encryptionkey@0"
I20260619 10:17:36.621860 8275 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781864224577058-825-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781864224577058-825-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781864224577058-825-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 10:17:36.628791 8275 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 10:17:36.631079 8275 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 10:17:36.631227 8275 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 10:17:36.635726 8275 rpc_server.cc:307] RPC server started. Bound to: 127.0.206.126:45657
I20260619 10:17:36.635767 8352 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.206.126:45657 every 8 connection(s)
I20260619 10:17:36.636063 8275 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781864224577058-825-0/minicluster-data/master-0/data/info.pb
I20260619 10:17:36.636602 8353 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 10:17:36.638836 8353 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f99ceb70a6684009b753712e007c5de5: Bootstrap starting.
I20260619 10:17:36.639094 825 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTU9OP6/build/release/bin/kudu as pid 8275
I20260619 10:17:36.639191 825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781864224577058-825-0/minicluster-data/master-0/wal/instance
I20260619 10:17:36.639392 825 external_mini_cluster.cc:1428] Setting key 157337b808fe4201397374d6d05f8813
I20260619 10:17:36.639495 8353 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f99ceb70a6684009b753712e007c5de5: Neither blocks nor log segments found. Creating new log.
I20260619 10:17:36.639798 8353 log.cc:824] T 00000000000000000000000000000000 P f99ceb70a6684009b753712e007c5de5: Log is configured to *not* fsync() on all Append() calls
I20260619 10:17:36.640525 8353 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f99ceb70a6684009b753712e007c5de5: No bootstrap required, opened a new log
I20260619 10:17:36.642313 8353 raft_consensus.cc:348] T 00000000000000000000000000000000 P f99ceb70a6684009b753712e007c5de5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f99ceb70a6684009b753712e007c5de5" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 45657 } }
I20260619 10:17:36.642481 8353 raft_consensus.cc:374] T 00000000000000000000000000000000 P f99ceb70a6684009b753712e007c5de5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 10:17:36.642513 8353 raft_consensus.cc:729] T 00000000000000000000000000000000 P f99ceb70a6684009b753712e007c5de5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f99ceb70a6684009b753712e007c5de5, State: Initialized, Role: FOLLOWER
I20260619 10:17:36.642652 8353 consensus_queue.cc:260] T 00000000000000000000000000000000 P f99ceb70a6684009b753712e007c5de5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f99ceb70a6684009b753712e007c5de5" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 45657 } }
I20260619 10:17:36.642728 8353 raft_consensus.cc:388] T 00000000000000000000000000000000 P f99ceb70a6684009b753712e007c5de5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 10:17:36.642765 8353 raft_consensus.cc:482] T 00000000000000000000000000000000 P f99ceb70a6684009b753712e007c5de5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 10:17:36.642802 8353 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f99ceb70a6684009b753712e007c5de5 [term 0 FOLLOWER]: Advancing to term 1
I20260619 10:17:36.643503 8353 raft_consensus.cc:504] T 00000000000000000000000000000000 P f99ceb70a6684009b753712e007c5de5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f99ceb70a6684009b753712e007c5de5" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 45657 } }
I20260619 10:17:36.643610 8353 leader_election.cc:302] T 00000000000000000000000000000000 P f99ceb70a6684009b753712e007c5de5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f99ceb70a6684009b753712e007c5de5; no voters:
I20260619 10:17:36.643759 8353 leader_election.cc:288] T 00000000000000000000000000000000 P f99ceb70a6684009b753712e007c5de5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 10:17:36.643990 8353 sys_catalog.cc:564] T 00000000000000000000000000000000 P f99ceb70a6684009b753712e007c5de5 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 10:17:36.644197 8358 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f99ceb70a6684009b753712e007c5de5 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 10:17:36.644359 8358 raft_consensus.cc:686] T 00000000000000000000000000000000 P f99ceb70a6684009b753712e007c5de5 [term 1 LEADER]: Becoming Leader. State: Replica: f99ceb70a6684009b753712e007c5de5, State: Running, Role: LEADER
I20260619 10:17:36.644435 8358 consensus_queue.cc:237] T 00000000000000000000000000000000 P f99ceb70a6684009b753712e007c5de5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f99ceb70a6684009b753712e007c5de5" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 45657 } }
I20260619 10:17:36.644731 8358 sys_catalog.cc:455] T 00000000000000000000000000000000 P f99ceb70a6684009b753712e007c5de5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f99ceb70a6684009b753712e007c5de5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f99ceb70a6684009b753712e007c5de5" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 45657 } } }
I20260619 10:17:36.644786 8358 sys_catalog.cc:458] T 00000000000000000000000000000000 P f99ceb70a6684009b753712e007c5de5 [sys.catalog]: This master's current role is: LEADER
I20260619 10:17:36.644884 8358 sys_catalog.cc:455] T 00000000000000000000000000000000 P f99ceb70a6684009b753712e007c5de5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f99ceb70a6684009b753712e007c5de5. Latest consensus state: current_term: 1 leader_uuid: "f99ceb70a6684009b753712e007c5de5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f99ceb70a6684009b753712e007c5de5" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 45657 } } }
I20260619 10:17:36.644930 8358 sys_catalog.cc:458] T 00000000000000000000000000000000 P f99ceb70a6684009b753712e007c5de5 [sys.catalog]: This master's current role is: LEADER
I20260619 10:17:36.645121 8367 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 10:17:36.645686 8367 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 10:17:36.647032 8375 mvcc.cc:205] Tried to move back new op lower bound from 7298515995224625152 to 7298515995215822848. Current Snapshot: MvccSnapshot[applied={T|T < 7298515995224625152}]
I20260619 10:17:36.647455 8367 catalog_manager.cc:1347] Generated new cluster ID: b2b86a1bf92941a2a224888556672c0f
I20260619 10:17:36.647506 8367 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Jun 19 10:17:36 dist-test-slave-6pt2 krb5kdc[8262](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864256, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.0.206.126@KRBTEST.COM
I20260619 10:17:36.651206 8367 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 10:17:36.651717 8356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:36.639857 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58380 (local address 127.0.206.126:45657)
0619 10:17:36.640037 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0619 10:17:36.640046 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 10:17:36.640080 (+ 34us) server_negotiation.cc:407] Connection header received
0619 10:17:36.640632 (+ 552us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:17:36.640642 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:17:36.640705 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:17:36.640850 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:17:36.641441 (+ 591us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:36.643219 (+ 1778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:36.644265 (+ 1046us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:36.644903 (+ 638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:36.645018 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:36.648027 (+ 3009us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 10:17:36.648032 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 10:17:36.648032 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 10:17:36.648043 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0619 10:17:36.650034 (+ 1991us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:17:36.650427 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:17:36.650430 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:17:36.650430 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:17:36.650532 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:17:36.650768 (+ 236us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:17:36.650769 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:17:36.650770 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:17:36.651078 (+ 308us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 10:17:36.651267 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0619 10:17:36.651357 (+ 90us) server_negotiation.cc:299] Negotiation successful
0619 10:17:36.651498 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":129,"thread_start_us":88,"threads_started":1}
I20260619 10:17:36.652096 8367 catalog_manager.cc:1504] Loading token signing keys...
I20260619 10:17:36.656670 8367 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f99ceb70a6684009b753712e007c5de5: Generated new TSK 0
I20260619 10:17:36.656819 8367 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.0.206.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.0.206.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.0.206.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781864224577058-825-0/krb5kdc/oryx_127.0.206.65.keytab.
Entry for principal oryx/127.0.206.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781864224577058-825-0/krb5kdc/oryx_127.0.206.65.keytab.
I20260619 10:17:36.684773 825 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTU9OP6/build/release/bin/kudu
/tmp/dist-test-taskTU9OP6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781864224577058-825-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781864224577058-825-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781864224577058-825-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781864224577058-825-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.206.65:0
--local_ip_for_outbound_sockets=127.0.206.65
--webserver_interface=127.0.206.65
--webserver_port=0
--tserver_master_addrs=127.0.206.126:45657
--keytab_file=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781864224577058-825-0/krb5kdc/oryx_127.0.206.65.keytab
--principal=oryx/127.0.206.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.206.84:41459
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781864224577058-825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781864224577058-825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781864224577058-825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 10:17:36.765291 8379 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 10:17:36.765493 8379 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 10:17:36.765517 8379 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 10:17:36.766659 8379 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 10:17:36.766705 8379 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 10:17:36.766726 8379 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260619 10:17:36.766742 8379 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.206.65
I20260619 10:17:36.768244 8379 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.206.84:41459
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781864224577058-825-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781864224577058-825-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781864224577058-825-0/krb5kdc/oryx_127.0.206.65.keytab
--principal=oryx/127.0.206.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.206.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781864224577058-825-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.206.65
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.0.206.126:45657
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8379
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.206.65
--log_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781864224577058-825-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 09:32:44 UTC on ccycloud.quasar-gbtldp.root.comops.site
build id 4455440
I20260619 10:17:36.768501 8379 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 10:17:36.768725 8379 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260619 10:17:36.774293 8379 server_base.cc:1034] running on GCE node
W20260619 10:17:36.774263 8384 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 10:17:36.774250 8387 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 10:17:36.774263 8385 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 10:17:36.774784 8379 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 10:17:36.775027 8379 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 10:17:36.776185 8379 hybrid_clock.cc:648] HybridClock initialized: now 1781864256776176 us; error 33 us; skew 500 ppm
Jun 19 10:17:36 dist-test-slave-6pt2 krb5kdc[8262](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864256, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.206.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 10:17:36.778515 8379 init.cc:377] Logged in from keytab as oryx/127.0.206.65@KRBTEST.COM (short username oryx)
I20260619 10:17:36.778957 8379 webserver.cc:460] Webserver started at http://127.0.206.65:46301/ using document root <none> and password file <none>
I20260619 10:17:36.779186 8379 fs_manager.cc:359] Metadata directory not provided
I20260619 10:17:36.779251 8379 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 10:17:36.779393 8379 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 10:17:36.780437 8379 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781864224577058-825-0/minicluster-data/ts-0/data/instance:
uuid: "0c4c7226ac8e48ca8d708ea8bf3b76f7"
format_stamp: "Formatted at 2026-06-19 10:17:36 on dist-test-slave-6pt2"
server_key: "05f1f918c516ea3e4f68c06a49e5d8ca"
server_key_iv: "161a0329a0ff7527db4fe36f08e82cc2"
server_key_version: "encryptionkey@0"
I20260619 10:17:36.780795 8379 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781864224577058-825-0/minicluster-data/ts-0/wal/instance:
uuid: "0c4c7226ac8e48ca8d708ea8bf3b76f7"
format_stamp: "Formatted at 2026-06-19 10:17:36 on dist-test-slave-6pt2"
server_key: "05f1f918c516ea3e4f68c06a49e5d8ca"
server_key_iv: "161a0329a0ff7527db4fe36f08e82cc2"
server_key_version: "encryptionkey@0"
I20260619 10:17:36.782078 8379 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 10:17:36.782773 8394 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 10:17:36.782939 8379 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 10:17:36.783051 8379 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781864224577058-825-0/minicluster-data/ts-0/data,/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781864224577058-825-0/minicluster-data/ts-0/wal
uuid: "0c4c7226ac8e48ca8d708ea8bf3b76f7"
format_stamp: "Formatted at 2026-06-19 10:17:36 on dist-test-slave-6pt2"
server_key: "05f1f918c516ea3e4f68c06a49e5d8ca"
server_key_iv: "161a0329a0ff7527db4fe36f08e82cc2"
server_key_version: "encryptionkey@0"
I20260619 10:17:36.783133 8379 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781864224577058-825-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781864224577058-825-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781864224577058-825-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 10:17:36.793288 8379 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 10:17:36.795148 8379 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 10:17:36.795298 8379 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 10:17:36.795794 8379 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 10:17:36.795857 8379 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 10:17:36.795917 8379 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 10:17:36.795940 8379 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 10:17:36.802776 8379 rpc_server.cc:307] RPC server started. Bound to: 127.0.206.65:37567
I20260619 10:17:36.802845 8509 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.206.65:37567 every 8 connection(s)
Jun 19 10:17:36 dist-test-slave-6pt2 krb5kdc[8262](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864256, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.206.65@KRBTEST.COM for oryx/127.0.206.126@KRBTEST.COM
I20260619 10:17:36.803162 8379 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781864224577058-825-0/minicluster-data/ts-0/data/info.pb
I20260619 10:17:36.805717 8356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:36.796273 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.65:55593 (local address 127.0.206.126:45657)
0619 10:17:36.796333 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0619 10:17:36.796336 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 10:17:36.796538 (+ 202us) server_negotiation.cc:407] Connection header received
0619 10:17:36.797006 (+ 468us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:17:36.797008 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:17:36.797023 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:17:36.797116 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:17:36.797933 (+ 817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:36.798795 (+ 862us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:36.800402 (+ 1607us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:36.800791 (+ 389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:36.800875 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:36.803568 (+ 2693us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 10:17:36.803570 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 10:17:36.803570 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 10:17:36.803577 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 10:17:36.804684 (+ 1107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:17:36.805029 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:17:36.805030 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:17:36.805030 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:17:36.805090 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:17:36.805253 (+ 163us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:17:36.805253 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:17:36.805254 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:17:36.805342 (+ 88us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 10:17:36.805388 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0619 10:17:36.805597 (+ 209us) server_negotiation.cc:299] Negotiation successful
0619 10:17:36.805657 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260619 10:17:36.805866 8403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:36.796247 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:45657 (local address 127.0.206.65:55593)
0619 10:17:36.796405 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0619 10:17:36.796442 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0619 10:17:36.796909 (+ 467us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 10:17:36.797168 (+ 259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 10:17:36.797171 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 10:17:36.797512 (+ 341us) client_negotiation.cc:189] Negotiated authn=SASL
0619 10:17:36.797811 (+ 299us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:36.797818 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:36.798884 (+ 1066us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:36.798885 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:36.800321 (+ 1436us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:36.800324 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:36.800885 (+ 561us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:36.800886 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:36.801031 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:36.801550 (+ 519us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 10:17:36.801561 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0619 10:17:36.803462 (+ 1901us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 10:17:36.804748 (+ 1286us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 10:17:36.804753 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 10:17:36.804755 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 10:17:36.804965 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 10:17:36.805129 (+ 164us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 10:17:36.805130 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 10:17:36.805130 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 10:17:36.805209 (+ 79us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 10:17:36.805393 (+ 184us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 10:17:36.805397 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 10:17:36.805497 (+ 100us) client_negotiation.cc:769] Sending connection context
0619 10:17:36.805583 (+ 86us) client_negotiation.cc:240] Negotiation successful
0619 10:17:36.805690 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":87,"threads_started":1}
I20260619 10:17:36.806298 8510 heartbeater.cc:347] Connected to a master server at 127.0.206.126:45657
I20260619 10:17:36.806376 8510 heartbeater.cc:464] Registering TS with master...
I20260619 10:17:36.806538 8510 heartbeater.cc:511] Master 127.0.206.126:45657 requested a full tablet report, sending...
I20260619 10:17:36.807103 8306 ts_manager.cc:194] Registered new tserver with Master: 0c4c7226ac8e48ca8d708ea8bf3b76f7 (127.0.206.65:37567)
I20260619 10:17:36.807925 8306 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.0.206.65@KRBTEST.COM'} at 127.0.206.65:55593
I20260619 10:17:36.809283 825 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTU9OP6/build/release/bin/kudu as pid 8379
I20260619 10:17:36.809376 825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781864224577058-825-0/minicluster-data/ts-0/wal/instance
I20260619 10:17:36.809610 825 external_mini_cluster.cc:1428] Setting key 2fdbd332ef3cc0146542ea4063cff2e0
I20260619 10:17:36.813017 8516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:36.807248 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:45657 (local address 127.0.206.65:41947)
0619 10:17:36.807382 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0619 10:17:36.807394 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 10:17:36.807444 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 10:17:36.807554 (+ 110us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 10:17:36.807555 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 10:17:36.807690 (+ 135us) client_negotiation.cc:189] Negotiated authn=SASL
0619 10:17:36.807731 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:36.807732 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:36.808410 (+ 678us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:36.808411 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:36.809193 (+ 782us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:36.809195 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:36.809568 (+ 373us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:36.809569 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:36.809630 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:36.810235 (+ 605us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 10:17:36.810239 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 10:17:36.810625 (+ 386us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 10:17:36.812093 (+ 1468us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 10:17:36.812094 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 10:17:36.812094 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 10:17:36.812254 (+ 160us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 10:17:36.812452 (+ 198us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 10:17:36.812453 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 10:17:36.812453 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 10:17:36.812508 (+ 55us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 10:17:36.812777 (+ 269us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 10:17:36.812777 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 10:17:36.812807 (+ 30us) client_negotiation.cc:769] Sending connection context
0619 10:17:36.812854 (+ 47us) client_negotiation.cc:240] Negotiation successful
0619 10:17:36.812953 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":64,"threads_started":1}
I20260619 10:17:36.813125 8356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:36.807404 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.65:41947 (local address 127.0.206.126:45657)
0619 10:17:36.807448 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0619 10:17:36.807450 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 10:17:36.807478 (+ 28us) server_negotiation.cc:407] Connection header received
0619 10:17:36.807497 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:17:36.807498 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:17:36.807507 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:17:36.807550 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:17:36.807787 (+ 237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:36.808337 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:36.809266 (+ 929us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:36.809485 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:36.809541 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:36.810689 (+ 1148us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 10:17:36.810690 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 10:17:36.810690 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 10:17:36.810696 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 10:17:36.812017 (+ 1321us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:17:36.812332 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:17:36.812333 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:17:36.812333 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:17:36.812386 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:17:36.812568 (+ 182us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:17:36.812569 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:17:36.812569 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:17:36.812718 (+ 149us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 10:17:36.812788 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0619 10:17:36.812892 (+ 104us) server_negotiation.cc:299] Negotiation successful
0619 10:17:36.812988 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
WARNING: no policy specified for oryx/127.0.206.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.0.206.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.0.206.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781864224577058-825-0/krb5kdc/oryx_127.0.206.66.keytab.
Entry for principal oryx/127.0.206.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781864224577058-825-0/krb5kdc/oryx_127.0.206.66.keytab.
I20260619 10:17:36.832484 825 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTU9OP6/build/release/bin/kudu
/tmp/dist-test-taskTU9OP6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781864224577058-825-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781864224577058-825-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781864224577058-825-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781864224577058-825-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.206.66:0
--local_ip_for_outbound_sockets=127.0.206.66
--webserver_interface=127.0.206.66
--webserver_port=0
--tserver_master_addrs=127.0.206.126:45657
--keytab_file=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781864224577058-825-0/krb5kdc/oryx_127.0.206.66.keytab
--principal=oryx/127.0.206.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.206.84:41459
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781864224577058-825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781864224577058-825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781864224577058-825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 10:17:36.911586 8519 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 10:17:36.911785 8519 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 10:17:36.911809 8519 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 10:17:36.912899 8519 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 10:17:36.912931 8519 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 10:17:36.912951 8519 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260619 10:17:36.912968 8519 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.206.66
I20260619 10:17:36.914331 8519 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.206.84:41459
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781864224577058-825-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781864224577058-825-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781864224577058-825-0/krb5kdc/oryx_127.0.206.66.keytab
--principal=oryx/127.0.206.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.206.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781864224577058-825-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.206.66
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.0.206.126:45657
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8519
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.206.66
--log_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781864224577058-825-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 09:32:44 UTC on ccycloud.quasar-gbtldp.root.comops.site
build id 4455440
I20260619 10:17:36.914515 8519 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 10:17:36.914743 8519 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 10:17:36.919731 8524 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 10:17:36.919723 8525 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 10:17:36.919826 8519 server_base.cc:1034] running on GCE node
W20260619 10:17:36.919723 8527 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 10:17:36.920226 8519 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 10:17:36.920454 8519 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 10:17:36.921595 8519 hybrid_clock.cc:648] HybridClock initialized: now 1781864256921576 us; error 35 us; skew 500 ppm
Jun 19 10:17:36 dist-test-slave-6pt2 krb5kdc[8262](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864256, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.206.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 10:17:36.924046 8519 init.cc:377] Logged in from keytab as oryx/127.0.206.66@KRBTEST.COM (short username oryx)
I20260619 10:17:36.924479 8519 webserver.cc:460] Webserver started at http://127.0.206.66:43677/ using document root <none> and password file <none>
I20260619 10:17:36.924726 8519 fs_manager.cc:359] Metadata directory not provided
I20260619 10:17:36.924772 8519 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 10:17:36.924913 8519 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 10:17:36.926046 8519 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781864224577058-825-0/minicluster-data/ts-1/data/instance:
uuid: "eed7c39853c04344aacc55afdd26f1fe"
format_stamp: "Formatted at 2026-06-19 10:17:36 on dist-test-slave-6pt2"
server_key: "756fa8687889a4cb269c6df0e7b48b7d"
server_key_iv: "534bf4ea8e4d7289838b025857eecd7f"
server_key_version: "encryptionkey@0"
I20260619 10:17:36.926403 8519 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781864224577058-825-0/minicluster-data/ts-1/wal/instance:
uuid: "eed7c39853c04344aacc55afdd26f1fe"
format_stamp: "Formatted at 2026-06-19 10:17:36 on dist-test-slave-6pt2"
server_key: "756fa8687889a4cb269c6df0e7b48b7d"
server_key_iv: "534bf4ea8e4d7289838b025857eecd7f"
server_key_version: "encryptionkey@0"
I20260619 10:17:36.927765 8519 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260619 10:17:36.928469 8534 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 10:17:36.928640 8519 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260619 10:17:36.928733 8519 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781864224577058-825-0/minicluster-data/ts-1/data,/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781864224577058-825-0/minicluster-data/ts-1/wal
uuid: "eed7c39853c04344aacc55afdd26f1fe"
format_stamp: "Formatted at 2026-06-19 10:17:36 on dist-test-slave-6pt2"
server_key: "756fa8687889a4cb269c6df0e7b48b7d"
server_key_iv: "534bf4ea8e4d7289838b025857eecd7f"
server_key_version: "encryptionkey@0"
I20260619 10:17:36.928813 8519 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781864224577058-825-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781864224577058-825-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781864224577058-825-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 10:17:36.940431 8519 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 10:17:36.942117 8519 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 10:17:36.942267 8519 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 10:17:36.942834 8519 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 10:17:36.942886 8519 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 10:17:36.942929 8519 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 10:17:36.942958 8519 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Jun 19 10:17:36 dist-test-slave-6pt2 krb5kdc[8262](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864256, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.206.66@KRBTEST.COM for oryx/127.0.206.126@KRBTEST.COM
I20260619 10:17:36.949857 8519 rpc_server.cc:307] RPC server started. Bound to: 127.0.206.66:36277
I20260619 10:17:36.949923 8649 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.206.66:36277 every 8 connection(s)
I20260619 10:17:36.950264 8519 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781864224577058-825-0/minicluster-data/ts-1/data/info.pb
I20260619 10:17:36.952767 8356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:36.943232 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.66:56055 (local address 127.0.206.126:45657)
0619 10:17:36.943278 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0619 10:17:36.943280 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 10:17:36.943493 (+ 213us) server_negotiation.cc:407] Connection header received
0619 10:17:36.943943 (+ 450us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:17:36.943945 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:17:36.943958 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:17:36.944039 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:17:36.944944 (+ 905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:36.945852 (+ 908us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:36.947169 (+ 1317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:36.947467 (+ 298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:36.947539 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:36.950403 (+ 2864us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 10:17:36.950405 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 10:17:36.950405 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 10:17:36.950415 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0619 10:17:36.951669 (+ 1254us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:17:36.952037 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:17:36.952038 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:17:36.952039 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:17:36.952095 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:17:36.952290 (+ 195us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:17:36.952291 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:17:36.952291 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:17:36.952388 (+ 97us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 10:17:36.952437 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0619 10:17:36.952650 (+ 213us) server_negotiation.cc:299] Negotiation successful
0619 10:17:36.952706 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260619 10:17:36.952931 8543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:36.943226 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:45657 (local address 127.0.206.66:56055)
0619 10:17:36.943357 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0619 10:17:36.943395 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0619 10:17:36.943854 (+ 459us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 10:17:36.944172 (+ 318us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 10:17:36.944176 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 10:17:36.944548 (+ 372us) client_negotiation.cc:189] Negotiated authn=SASL
0619 10:17:36.944839 (+ 291us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:36.944845 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:36.945944 (+ 1099us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:36.945945 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:36.947078 (+ 1133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:36.947081 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:36.947557 (+ 476us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:36.947558 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:36.947699 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:36.948297 (+ 598us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 10:17:36.948315 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0619 10:17:36.950289 (+ 1974us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 10:17:36.951736 (+ 1447us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 10:17:36.951741 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 10:17:36.951743 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 10:17:36.951972 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 10:17:36.952149 (+ 177us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 10:17:36.952149 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 10:17:36.952150 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 10:17:36.952241 (+ 91us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 10:17:36.952436 (+ 195us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 10:17:36.952440 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 10:17:36.952550 (+ 110us) client_negotiation.cc:769] Sending connection context
0619 10:17:36.952639 (+ 89us) client_negotiation.cc:240] Negotiation successful
0619 10:17:36.952753 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":94,"thread_start_us":71,"threads_started":1}
I20260619 10:17:36.953414 8650 heartbeater.cc:347] Connected to a master server at 127.0.206.126:45657
I20260619 10:17:36.953503 8650 heartbeater.cc:464] Registering TS with master...
I20260619 10:17:36.953675 8650 heartbeater.cc:511] Master 127.0.206.126:45657 requested a full tablet report, sending...
I20260619 10:17:36.954088 8307 ts_manager.cc:194] Registered new tserver with Master: eed7c39853c04344aacc55afdd26f1fe (127.0.206.66:36277)
I20260619 10:17:36.954635 8307 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.0.206.66@KRBTEST.COM'} at 127.0.206.66:56055
I20260619 10:17:36.956893 825 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTU9OP6/build/release/bin/kudu as pid 8519
I20260619 10:17:36.957013 825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781864224577058-825-0/minicluster-data/ts-1/wal/instance
I20260619 10:17:36.957221 825 external_mini_cluster.cc:1428] Setting key 5f45824252a38ee10cb647dacd9ea157
I20260619 10:17:36.960121 8356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:36.954418 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.66:47081 (local address 127.0.206.126:45657)
0619 10:17:36.954458 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0619 10:17:36.954460 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 10:17:36.954522 (+ 62us) server_negotiation.cc:407] Connection header received
0619 10:17:36.954574 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:17:36.954574 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:17:36.954582 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:17:36.954668 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:17:36.955009 (+ 341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:36.955653 (+ 644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:36.956300 (+ 647us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:36.956468 (+ 168us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:36.956514 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:36.957473 (+ 959us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 10:17:36.957475 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 10:17:36.957475 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 10:17:36.957482 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 10:17:36.959023 (+ 1541us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:17:36.959392 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:17:36.959393 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:17:36.959393 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:17:36.959460 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:17:36.959669 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:17:36.959669 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:17:36.959670 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:17:36.959795 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 10:17:36.959848 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0619 10:17:36.960003 (+ 155us) server_negotiation.cc:299] Negotiation successful
0619 10:17:36.960063 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260619 10:17:36.960232 8656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:36.954337 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:45657 (local address 127.0.206.66:47081)
0619 10:17:36.954463 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0619 10:17:36.954474 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 10:17:36.954530 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 10:17:36.954671 (+ 141us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 10:17:36.954672 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 10:17:36.954866 (+ 194us) client_negotiation.cc:189] Negotiated authn=SASL
0619 10:17:36.954930 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:36.954931 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:36.955732 (+ 801us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:36.955732 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:36.956224 (+ 492us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:36.956225 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:36.956523 (+ 298us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:36.956523 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:36.956568 (+ 45us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:36.957038 (+ 470us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 10:17:36.957043 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 10:17:36.957397 (+ 354us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 10:17:36.959124 (+ 1727us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 10:17:36.959125 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 10:17:36.959125 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 10:17:36.959300 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 10:17:36.959532 (+ 232us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 10:17:36.959533 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 10:17:36.959534 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 10:17:36.959608 (+ 74us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 10:17:36.959855 (+ 247us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 10:17:36.959855 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 10:17:36.959901 (+ 46us) client_negotiation.cc:769] Sending connection context
0619 10:17:36.959985 (+ 84us) client_negotiation.cc:240] Negotiation successful
0619 10:17:36.960094 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":97,"thread_start_us":72,"threads_started":1}
WARNING: no policy specified for oryx/127.0.206.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.0.206.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.0.206.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781864224577058-825-0/krb5kdc/oryx_127.0.206.67.keytab.
Entry for principal oryx/127.0.206.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781864224577058-825-0/krb5kdc/oryx_127.0.206.67.keytab.
I20260619 10:17:36.980875 825 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTU9OP6/build/release/bin/kudu
/tmp/dist-test-taskTU9OP6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781864224577058-825-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781864224577058-825-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781864224577058-825-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781864224577058-825-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.206.67:0
--local_ip_for_outbound_sockets=127.0.206.67
--webserver_interface=127.0.206.67
--webserver_port=0
--tserver_master_addrs=127.0.206.126:45657
--keytab_file=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781864224577058-825-0/krb5kdc/oryx_127.0.206.67.keytab
--principal=oryx/127.0.206.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.206.84:41459
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781864224577058-825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781864224577058-825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781864224577058-825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 10:17:37.061280 8659 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 10:17:37.061501 8659 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 10:17:37.061525 8659 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 10:17:37.062739 8659 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 10:17:37.062774 8659 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 10:17:37.062795 8659 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260619 10:17:37.062812 8659 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.206.67
I20260619 10:17:37.064188 8659 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.206.84:41459
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781864224577058-825-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781864224577058-825-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781864224577058-825-0/krb5kdc/oryx_127.0.206.67.keytab
--principal=oryx/127.0.206.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.206.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781864224577058-825-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.206.67
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.0.206.126:45657
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8659
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.206.67
--log_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781864224577058-825-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 09:32:44 UTC on ccycloud.quasar-gbtldp.root.comops.site
build id 4455440
I20260619 10:17:37.064365 8659 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 10:17:37.064589 8659 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 10:17:37.069274 8667 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 10:17:37.069289 8665 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 10:17:37.069280 8664 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 10:17:37.069599 8659 server_base.cc:1034] running on GCE node
I20260619 10:17:37.069799 8659 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 10:17:37.070009 8659 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 10:17:37.071185 8659 hybrid_clock.cc:648] HybridClock initialized: now 1781864257071179 us; error 33 us; skew 500 ppm
Jun 19 10:17:37 dist-test-slave-6pt2 krb5kdc[8262](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864257, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.206.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 10:17:37.073510 8659 init.cc:377] Logged in from keytab as oryx/127.0.206.67@KRBTEST.COM (short username oryx)
I20260619 10:17:37.073958 8659 webserver.cc:460] Webserver started at http://127.0.206.67:46367/ using document root <none> and password file <none>
I20260619 10:17:37.074187 8659 fs_manager.cc:359] Metadata directory not provided
I20260619 10:17:37.074251 8659 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 10:17:37.074378 8659 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 10:17:37.075436 8659 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781864224577058-825-0/minicluster-data/ts-2/data/instance:
uuid: "0e4f629e966e419fb21b57161abdf46c"
format_stamp: "Formatted at 2026-06-19 10:17:37 on dist-test-slave-6pt2"
server_key: "05e42491c59ee6a334ee6063dc8de20e"
server_key_iv: "afcedaeb55383c7045ee9ddfc2170de8"
server_key_version: "encryptionkey@0"
I20260619 10:17:37.075834 8659 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781864224577058-825-0/minicluster-data/ts-2/wal/instance:
uuid: "0e4f629e966e419fb21b57161abdf46c"
format_stamp: "Formatted at 2026-06-19 10:17:37 on dist-test-slave-6pt2"
server_key: "05e42491c59ee6a334ee6063dc8de20e"
server_key_iv: "afcedaeb55383c7045ee9ddfc2170de8"
server_key_version: "encryptionkey@0"
I20260619 10:17:37.077092 8659 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260619 10:17:37.077764 8674 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 10:17:37.077919 8659 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260619 10:17:37.078018 8659 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781864224577058-825-0/minicluster-data/ts-2/data,/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781864224577058-825-0/minicluster-data/ts-2/wal
uuid: "0e4f629e966e419fb21b57161abdf46c"
format_stamp: "Formatted at 2026-06-19 10:17:37 on dist-test-slave-6pt2"
server_key: "05e42491c59ee6a334ee6063dc8de20e"
server_key_iv: "afcedaeb55383c7045ee9ddfc2170de8"
server_key_version: "encryptionkey@0"
I20260619 10:17:37.078086 8659 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781864224577058-825-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781864224577058-825-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781864224577058-825-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 10:17:37.098855 8659 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 10:17:37.100703 8659 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 10:17:37.100863 8659 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 10:17:37.101480 8659 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 10:17:37.101531 8659 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 10:17:37.101567 8659 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 10:17:37.101614 8659 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Jun 19 10:17:37 dist-test-slave-6pt2 krb5kdc[8262](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864257, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.206.67@KRBTEST.COM for oryx/127.0.206.126@KRBTEST.COM
I20260619 10:17:37.108608 8659 rpc_server.cc:307] RPC server started. Bound to: 127.0.206.67:42531
I20260619 10:17:37.108723 8789 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.206.67:42531 every 8 connection(s)
I20260619 10:17:37.109031 8659 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781864224577058-825-0/minicluster-data/ts-2/data/info.pb
I20260619 10:17:37.111497 8356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:37.101859 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.67:48715 (local address 127.0.206.126:45657)
0619 10:17:37.101904 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0619 10:17:37.101906 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 10:17:37.102181 (+ 275us) server_negotiation.cc:407] Connection header received
0619 10:17:37.102776 (+ 595us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:17:37.102778 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:17:37.102791 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:17:37.102847 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:17:37.103501 (+ 654us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:37.103996 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:37.105069 (+ 1073us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:37.105252 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:37.105336 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:37.108503 (+ 3167us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 10:17:37.108505 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 10:17:37.108505 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 10:17:37.108512 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 10:17:37.110116 (+ 1604us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:17:37.110524 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:17:37.110525 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:17:37.110525 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:17:37.110586 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:17:37.110852 (+ 266us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:17:37.110853 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:17:37.110853 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:17:37.110973 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 10:17:37.111055 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0619 10:17:37.111319 (+ 264us) server_negotiation.cc:299] Negotiation successful
0619 10:17:37.111408 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260619 10:17:37.111690 8683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:37.101871 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:45657 (local address 127.0.206.67:48715)
0619 10:17:37.102045 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0619 10:17:37.102085 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0619 10:17:37.102670 (+ 585us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 10:17:37.102921 (+ 251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 10:17:37.102924 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 10:17:37.103223 (+ 299us) client_negotiation.cc:189] Negotiated authn=SASL
0619 10:17:37.103406 (+ 183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:37.103411 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:37.104106 (+ 695us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:37.104107 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:37.104923 (+ 816us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:37.104925 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:37.105346 (+ 421us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:37.105347 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:37.105470 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:37.106058 (+ 588us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 10:17:37.106069 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0619 10:17:37.108385 (+ 2316us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 10:17:37.110206 (+ 1821us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 10:17:37.110212 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 10:17:37.110214 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 10:17:37.110435 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 10:17:37.110667 (+ 232us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 10:17:37.110668 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 10:17:37.110669 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 10:17:37.110785 (+ 116us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 10:17:37.111045 (+ 260us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 10:17:37.111051 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 10:17:37.111191 (+ 140us) client_negotiation.cc:769] Sending connection context
0619 10:17:37.111297 (+ 106us) client_negotiation.cc:240] Negotiation successful
0619 10:17:37.111455 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":99,"threads_started":1}
I20260619 10:17:37.112165 8790 heartbeater.cc:347] Connected to a master server at 127.0.206.126:45657
I20260619 10:17:37.112244 8790 heartbeater.cc:464] Registering TS with master...
I20260619 10:17:37.112412 8790 heartbeater.cc:511] Master 127.0.206.126:45657 requested a full tablet report, sending...
I20260619 10:17:37.112802 8307 ts_manager.cc:194] Registered new tserver with Master: 0e4f629e966e419fb21b57161abdf46c (127.0.206.67:42531)
I20260619 10:17:37.113308 8307 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.0.206.67@KRBTEST.COM'} at 127.0.206.67:48715
I20260619 10:17:37.115270 825 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTU9OP6/build/release/bin/kudu as pid 8659
I20260619 10:17:37.115382 825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781864224577058-825-0/minicluster-data/ts-2/wal/instance
I20260619 10:17:37.115636 825 external_mini_cluster.cc:1428] Setting key 2fce0ebbefb4cc891ec44a49f6a7c824
I20260619 10:17:37.116557 825 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 10:17:37.118748 8356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:37.113185 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.67:55253 (local address 127.0.206.126:45657)
0619 10:17:37.113220 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0619 10:17:37.113221 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 10:17:37.113341 (+ 120us) server_negotiation.cc:407] Connection header received
0619 10:17:37.113410 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:17:37.113411 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:17:37.113421 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:17:37.113464 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:17:37.113836 (+ 372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:37.114371 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:37.115101 (+ 730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:37.115397 (+ 296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:37.115479 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:37.116369 (+ 890us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 10:17:37.116370 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 10:17:37.116370 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 10:17:37.116376 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 10:17:37.117601 (+ 1225us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:17:37.117934 (+ 333us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:17:37.117934 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:17:37.117935 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:17:37.117985 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:17:37.118205 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:17:37.118207 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:17:37.118207 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:17:37.118356 (+ 149us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 10:17:37.118423 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0619 10:17:37.118584 (+ 161us) server_negotiation.cc:299] Negotiation successful
0619 10:17:37.118691 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260619 10:17:37.118817 8796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:37.113130 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:45657 (local address 127.0.206.67:55253)
0619 10:17:37.113279 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0619 10:17:37.113288 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0619 10:17:37.113354 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 10:17:37.113497 (+ 143us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 10:17:37.113498 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 10:17:37.113700 (+ 202us) client_negotiation.cc:189] Negotiated authn=SASL
0619 10:17:37.113751 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:37.113753 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:37.114469 (+ 716us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:37.114470 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:37.115025 (+ 555us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:37.115027 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:37.115479 (+ 452us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:37.115480 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:37.115531 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:37.115931 (+ 400us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 10:17:37.115935 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 10:17:37.116297 (+ 362us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 10:17:37.117683 (+ 1386us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 10:17:37.117684 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 10:17:37.117684 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 10:17:37.117851 (+ 167us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 10:17:37.118053 (+ 202us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 10:17:37.118055 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 10:17:37.118055 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 10:17:37.118123 (+ 68us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 10:17:37.118430 (+ 307us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 10:17:37.118431 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 10:17:37.118481 (+ 50us) client_negotiation.cc:769] Sending connection context
0619 10:17:37.118557 (+ 76us) client_negotiation.cc:240] Negotiation successful
0619 10:17:37.118683 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":66,"threads_started":1}
Jun 19 10:17:37 dist-test-slave-6pt2 krb5kdc[8262](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864257, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Jun 19 10:17:37 dist-test-slave-6pt2 krb5kdc[8262](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.0.206.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 10:17:37 dist-test-slave-6pt2 krb5kdc[8262](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.0.206.126@KRBTEST.COM, Server not found in Kerberos database
W20260619 10:17:37.138515 8356 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 10:17:37.133108 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58392 (local address 127.0.206.126:45657)
0619 10:17:37.133175 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0619 10:17:37.133177 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 10:17:37.133187 (+ 10us) server_negotiation.cc:407] Connection header received
0619 10:17:37.133251 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:17:37.133253 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:17:37.133267 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:17:37.133327 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:17:37.133903 (+ 576us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:37.134403 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:37.135238 (+ 835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:37.135429 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:37.135511 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:37.138422 (+ 2911us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58392: BlockingRecv error: recv got EOF from 127.0.0.1:58392 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
Jun 19 10:17:37 dist-test-slave-6pt2 krb5kdc[8262](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864257, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.0.206.126@KRBTEST.COM
I20260619 10:17:37.146030 8356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:37.139311 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58402 (local address 127.0.206.126:45657)
0619 10:17:37.139369 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0619 10:17:37.139371 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 10:17:37.139411 (+ 40us) server_negotiation.cc:407] Connection header received
0619 10:17:37.139464 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:17:37.139464 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:17:37.139472 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:17:37.139515 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:17:37.139936 (+ 421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:37.140359 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:37.141060 (+ 701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:37.141227 (+ 167us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:37.141281 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:37.143440 (+ 2159us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 10:17:37.143442 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 10:17:37.143442 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 10:17:37.143449 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 10:17:37.144941 (+ 1492us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:17:37.145342 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:17:37.145343 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:17:37.145344 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:17:37.145397 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:17:37.145613 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:17:37.145614 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:17:37.145614 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:17:37.145712 (+ 98us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 10:17:37.145811 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0619 10:17:37.145908 (+ 97us) server_negotiation.cc:299] Negotiation successful
0619 10:17:37.145966 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260619 10:17:37.147666 8307 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:58402:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 10:17:37.149693 8307 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 10:17:37.156721 8810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:37.151858 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.65:37567 (local address 127.0.0.1:43058)
0619 10:17:37.152192 (+ 334us) negotiation.cc:107] Waiting for socket to connect
0619 10:17:37.152201 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0619 10:17:37.152252 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 10:17:37.152454 (+ 202us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 10:17:37.152457 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 10:17:37.152459 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 10:17:37.152537 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:37.152539 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:37.154036 (+ 1497us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:37.154037 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:37.155479 (+ 1442us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:37.155480 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:37.156415 (+ 935us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:37.156416 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:37.156489 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:37.156499 (+ 10us) client_negotiation.cc:769] Sending connection context
0619 10:17:37.156581 (+ 82us) client_negotiation.cc:240] Negotiation successful
0619 10:17:37.156634 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":315,"mutex_wait_us":42,"thread_start_us":55,"threads_started":1}
I20260619 10:17:37.156735 8813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:37.152023 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43058 (local address 127.0.206.65:37567)
0619 10:17:37.152185 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0619 10:17:37.152190 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 10:17:37.152247 (+ 57us) server_negotiation.cc:407] Connection header received
0619 10:17:37.152331 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:17:37.152334 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:17:37.152353 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:17:37.152456 (+ 103us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 10:17:37.152641 (+ 185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:37.153969 (+ 1328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:37.155561 (+ 1592us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:37.156329 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:37.156417 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:37.156493 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0619 10:17:37.156598 (+ 105us) server_negotiation.cc:299] Negotiation successful
0619 10:17:37.156641 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":124,"thread_start_us":85,"threads_started":1}
I20260619 10:17:37.157454 8444 tablet_service.cc:1467] Processing CreateTablet for tablet 098e55193002499d98f043abb96af42b (DEFAULT_TABLE table=test-table [id=3094b700c8f84a6195757f172bb1466c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 10:17:37.157805 8444 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 098e55193002499d98f043abb96af42b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 10:17:37.157984 8809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:37.151825 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.66:36277 (local address 127.0.0.1:36298)
0619 10:17:37.152129 (+ 304us) negotiation.cc:107] Waiting for socket to connect
0619 10:17:37.152170 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0619 10:17:37.152238 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 10:17:37.152627 (+ 389us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 10:17:37.152628 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 10:17:37.152631 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 10:17:37.152685 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:37.152687 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:37.154033 (+ 1346us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:37.154034 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:37.155477 (+ 1443us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:37.155479 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:37.157699 (+ 2220us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:37.157700 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:37.157784 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:37.157787 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 10:17:37.157837 (+ 50us) client_negotiation.cc:240] Negotiation successful
0619 10:17:37.157883 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":273,"mutex_wait_us":53,"thread_start_us":53,"threads_started":1}
I20260619 10:17:37.158099 8811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:37.152012 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.67:42531 (local address 127.0.0.1:36834)
0619 10:17:37.154067 (+ 2055us) negotiation.cc:107] Waiting for socket to connect
0619 10:17:37.154078 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 10:17:37.154132 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 10:17:37.154314 (+ 182us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 10:17:37.154314 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 10:17:37.154316 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 10:17:37.154356 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:37.154358 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:37.155418 (+ 1060us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:37.155420 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:37.156583 (+ 1163us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:37.156585 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:37.157777 (+ 1192us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:37.157778 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:37.157840 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:37.157843 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 10:17:37.157901 (+ 58us) client_negotiation.cc:240] Negotiation successful
0619 10:17:37.157936 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2041,"thread_start_us":47,"threads_started":1}
I20260619 10:17:37.158214 8814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:37.152195 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36834 (local address 127.0.206.67:42531)
0619 10:17:37.152577 (+ 382us) server_negotiation.cc:206] Beginning negotiation
0619 10:17:37.152582 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 10:17:37.154148 (+ 1566us) server_negotiation.cc:407] Connection header received
0619 10:17:37.154208 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:17:37.154210 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:17:37.154226 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:17:37.154318 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 10:17:37.154453 (+ 135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:37.155312 (+ 859us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:37.156674 (+ 1362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:37.157604 (+ 930us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:37.157669 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:37.157747 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0619 10:17:37.158028 (+ 281us) server_negotiation.cc:299] Negotiation successful
0619 10:17:37.158074 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":350,"thread_start_us":87,"threads_started":1}
I20260619 10:17:37.158824 8724 tablet_service.cc:1467] Processing CreateTablet for tablet 098e55193002499d98f043abb96af42b (DEFAULT_TABLE table=test-table [id=3094b700c8f84a6195757f172bb1466c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 10:17:37.159142 8724 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 098e55193002499d98f043abb96af42b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 10:17:37.161233 8815 tablet_bootstrap.cc:492] T 098e55193002499d98f043abb96af42b P 0c4c7226ac8e48ca8d708ea8bf3b76f7: Bootstrap starting.
I20260619 10:17:37.161782 8815 tablet_bootstrap.cc:654] T 098e55193002499d98f043abb96af42b P 0c4c7226ac8e48ca8d708ea8bf3b76f7: Neither blocks nor log segments found. Creating new log.
I20260619 10:17:37.162065 8815 log.cc:824] T 098e55193002499d98f043abb96af42b P 0c4c7226ac8e48ca8d708ea8bf3b76f7: Log is configured to *not* fsync() on all Append() calls
I20260619 10:17:37.162348 8816 tablet_bootstrap.cc:492] T 098e55193002499d98f043abb96af42b P 0e4f629e966e419fb21b57161abdf46c: Bootstrap starting.
I20260619 10:17:37.162633 8812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:37.151986 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36298 (local address 127.0.206.66:36277)
0619 10:17:37.152406 (+ 420us) server_negotiation.cc:206] Beginning negotiation
0619 10:17:37.152411 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 10:17:37.152421 (+ 10us) server_negotiation.cc:407] Connection header received
0619 10:17:37.152459 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:17:37.152461 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:17:37.152476 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:17:37.152557 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 10:17:37.152769 (+ 212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:37.153943 (+ 1174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:37.156528 (+ 2585us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:37.157505 (+ 977us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:37.157580 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:37.157661 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0619 10:17:37.158217 (+ 556us) server_negotiation.cc:299] Negotiation successful
0619 10:17:37.162477 (+ 4260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":384,"thread_start_us":112,"threads_started":1}
I20260619 10:17:37.162945 8815 tablet_bootstrap.cc:492] T 098e55193002499d98f043abb96af42b P 0c4c7226ac8e48ca8d708ea8bf3b76f7: No bootstrap required, opened a new log
I20260619 10:17:37.163019 8815 ts_tablet_manager.cc:1397] T 098e55193002499d98f043abb96af42b P 0c4c7226ac8e48ca8d708ea8bf3b76f7: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260619 10:17:37.163057 8816 tablet_bootstrap.cc:654] T 098e55193002499d98f043abb96af42b P 0e4f629e966e419fb21b57161abdf46c: Neither blocks nor log segments found. Creating new log.
I20260619 10:17:37.163280 8584 tablet_service.cc:1467] Processing CreateTablet for tablet 098e55193002499d98f043abb96af42b (DEFAULT_TABLE table=test-table [id=3094b700c8f84a6195757f172bb1466c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 10:17:37.163378 8816 log.cc:824] T 098e55193002499d98f043abb96af42b P 0e4f629e966e419fb21b57161abdf46c: Log is configured to *not* fsync() on all Append() calls
I20260619 10:17:37.163616 8584 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 098e55193002499d98f043abb96af42b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 10:17:37.164716 8815 raft_consensus.cc:348] T 098e55193002499d98f043abb96af42b P 0c4c7226ac8e48ca8d708ea8bf3b76f7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eed7c39853c04344aacc55afdd26f1fe" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 36277 } } peers { permanent_uuid: "0c4c7226ac8e48ca8d708ea8bf3b76f7" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 37567 } } peers { permanent_uuid: "0e4f629e966e419fb21b57161abdf46c" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 42531 } }
I20260619 10:17:37.164844 8815 raft_consensus.cc:374] T 098e55193002499d98f043abb96af42b P 0c4c7226ac8e48ca8d708ea8bf3b76f7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 10:17:37.164865 8815 raft_consensus.cc:729] T 098e55193002499d98f043abb96af42b P 0c4c7226ac8e48ca8d708ea8bf3b76f7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0c4c7226ac8e48ca8d708ea8bf3b76f7, State: Initialized, Role: FOLLOWER
I20260619 10:17:37.164980 8815 consensus_queue.cc:260] T 098e55193002499d98f043abb96af42b P 0c4c7226ac8e48ca8d708ea8bf3b76f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eed7c39853c04344aacc55afdd26f1fe" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 36277 } } peers { permanent_uuid: "0c4c7226ac8e48ca8d708ea8bf3b76f7" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 37567 } } peers { permanent_uuid: "0e4f629e966e419fb21b57161abdf46c" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 42531 } }
I20260619 10:17:37.165257 8815 ts_tablet_manager.cc:1428] T 098e55193002499d98f043abb96af42b P 0c4c7226ac8e48ca8d708ea8bf3b76f7: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260619 10:17:37.165320 8510 heartbeater.cc:503] Master 127.0.206.126:45657 was elected leader, sending a full tablet report...
I20260619 10:17:37.165843 8816 tablet_bootstrap.cc:492] T 098e55193002499d98f043abb96af42b P 0e4f629e966e419fb21b57161abdf46c: No bootstrap required, opened a new log
I20260619 10:17:37.165894 8816 ts_tablet_manager.cc:1397] T 098e55193002499d98f043abb96af42b P 0e4f629e966e419fb21b57161abdf46c: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20260619 10:17:37.167213 8820 tablet_bootstrap.cc:492] T 098e55193002499d98f043abb96af42b P eed7c39853c04344aacc55afdd26f1fe: Bootstrap starting.
I20260619 10:17:37.167817 8820 tablet_bootstrap.cc:654] T 098e55193002499d98f043abb96af42b P eed7c39853c04344aacc55afdd26f1fe: Neither blocks nor log segments found. Creating new log.
I20260619 10:17:37.167713 8816 raft_consensus.cc:348] T 098e55193002499d98f043abb96af42b P 0e4f629e966e419fb21b57161abdf46c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eed7c39853c04344aacc55afdd26f1fe" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 36277 } } peers { permanent_uuid: "0c4c7226ac8e48ca8d708ea8bf3b76f7" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 37567 } } peers { permanent_uuid: "0e4f629e966e419fb21b57161abdf46c" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 42531 } }
I20260619 10:17:37.167869 8816 raft_consensus.cc:374] T 098e55193002499d98f043abb96af42b P 0e4f629e966e419fb21b57161abdf46c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 10:17:37.167906 8816 raft_consensus.cc:729] T 098e55193002499d98f043abb96af42b P 0e4f629e966e419fb21b57161abdf46c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e4f629e966e419fb21b57161abdf46c, State: Initialized, Role: FOLLOWER
I20260619 10:17:37.168064 8816 consensus_queue.cc:260] T 098e55193002499d98f043abb96af42b P 0e4f629e966e419fb21b57161abdf46c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eed7c39853c04344aacc55afdd26f1fe" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 36277 } } peers { permanent_uuid: "0c4c7226ac8e48ca8d708ea8bf3b76f7" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 37567 } } peers { permanent_uuid: "0e4f629e966e419fb21b57161abdf46c" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 42531 } }
I20260619 10:17:37.168131 8820 log.cc:824] T 098e55193002499d98f043abb96af42b P eed7c39853c04344aacc55afdd26f1fe: Log is configured to *not* fsync() on all Append() calls
I20260619 10:17:37.168344 8816 ts_tablet_manager.cc:1428] T 098e55193002499d98f043abb96af42b P 0e4f629e966e419fb21b57161abdf46c: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260619 10:17:37.168371 8790 heartbeater.cc:503] Master 127.0.206.126:45657 was elected leader, sending a full tablet report...
I20260619 10:17:37.169173 8820 tablet_bootstrap.cc:492] T 098e55193002499d98f043abb96af42b P eed7c39853c04344aacc55afdd26f1fe: No bootstrap required, opened a new log
I20260619 10:17:37.169255 8820 ts_tablet_manager.cc:1397] T 098e55193002499d98f043abb96af42b P eed7c39853c04344aacc55afdd26f1fe: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260619 10:17:37.170925 8820 raft_consensus.cc:348] T 098e55193002499d98f043abb96af42b P eed7c39853c04344aacc55afdd26f1fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eed7c39853c04344aacc55afdd26f1fe" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 36277 } } peers { permanent_uuid: "0c4c7226ac8e48ca8d708ea8bf3b76f7" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 37567 } } peers { permanent_uuid: "0e4f629e966e419fb21b57161abdf46c" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 42531 } }
I20260619 10:17:37.171051 8820 raft_consensus.cc:374] T 098e55193002499d98f043abb96af42b P eed7c39853c04344aacc55afdd26f1fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 10:17:37.171094 8820 raft_consensus.cc:729] T 098e55193002499d98f043abb96af42b P eed7c39853c04344aacc55afdd26f1fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eed7c39853c04344aacc55afdd26f1fe, State: Initialized, Role: FOLLOWER
I20260619 10:17:37.171214 8820 consensus_queue.cc:260] T 098e55193002499d98f043abb96af42b P eed7c39853c04344aacc55afdd26f1fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eed7c39853c04344aacc55afdd26f1fe" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 36277 } } peers { permanent_uuid: "0c4c7226ac8e48ca8d708ea8bf3b76f7" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 37567 } } peers { permanent_uuid: "0e4f629e966e419fb21b57161abdf46c" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 42531 } }
I20260619 10:17:37.171480 8820 ts_tablet_manager.cc:1428] T 098e55193002499d98f043abb96af42b P eed7c39853c04344aacc55afdd26f1fe: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260619 10:17:37.171505 8650 heartbeater.cc:503] Master 127.0.206.126:45657 was elected leader, sending a full tablet report...
W20260619 10:17:37.200996 8651 tablet.cc:2369] T 098e55193002499d98f043abb96af42b P eed7c39853c04344aacc55afdd26f1fe: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 10:17:37.269585 8819 raft_consensus.cc:482] T 098e55193002499d98f043abb96af42b P 0c4c7226ac8e48ca8d708ea8bf3b76f7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 10:17:37.269701 8819 raft_consensus.cc:504] T 098e55193002499d98f043abb96af42b P 0c4c7226ac8e48ca8d708ea8bf3b76f7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eed7c39853c04344aacc55afdd26f1fe" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 36277 } } peers { permanent_uuid: "0c4c7226ac8e48ca8d708ea8bf3b76f7" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 37567 } } peers { permanent_uuid: "0e4f629e966e419fb21b57161abdf46c" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 42531 } }
I20260619 10:17:37.270102 8819 leader_election.cc:288] T 098e55193002499d98f043abb96af42b P 0c4c7226ac8e48ca8d708ea8bf3b76f7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eed7c39853c04344aacc55afdd26f1fe (127.0.206.66:36277), 0e4f629e966e419fb21b57161abdf46c (127.0.206.67:42531)
I20260619 10:17:37.272778 8403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:37.270283 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.66:36277 (local address 127.0.206.65:46055)
0619 10:17:37.270336 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0619 10:17:37.270348 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 10:17:37.270389 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 10:17:37.270581 (+ 192us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 10:17:37.270582 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 10:17:37.270583 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 10:17:37.270668 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:37.270670 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:37.271272 (+ 602us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:37.271273 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:37.272023 (+ 750us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:37.272024 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:37.272526 (+ 502us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:37.272527 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:37.272603 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:37.272605 (+ 2us) client_negotiation.cc:769] Sending connection context
0619 10:17:37.272673 (+ 68us) client_negotiation.cc:240] Negotiation successful
0619 10:17:37.272712 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":31}
I20260619 10:17:37.272778 8812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:37.270360 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.65:46055 (local address 127.0.206.66:36277)
0619 10:17:37.270423 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0619 10:17:37.270425 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 10:17:37.270436 (+ 11us) server_negotiation.cc:407] Connection header received
0619 10:17:37.270466 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:17:37.270467 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:17:37.270476 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:17:37.270561 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 10:17:37.270737 (+ 176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:37.271193 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:37.272104 (+ 911us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:37.272451 (+ 347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:37.272528 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:37.272536 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0619 10:17:37.272681 (+ 145us) server_negotiation.cc:299] Negotiation successful
0619 10:17:37.272717 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260619 10:17:37.272964 8814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:37.270524 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.65:33079 (local address 127.0.206.67:42531)
0619 10:17:37.270571 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0619 10:17:37.270572 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 10:17:37.270670 (+ 98us) server_negotiation.cc:407] Connection header received
0619 10:17:37.270717 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:17:37.270717 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:17:37.270725 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:17:37.270788 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 10:17:37.270938 (+ 150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:37.271372 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:37.272277 (+ 905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:37.272620 (+ 343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:37.272695 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:37.272702 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0619 10:17:37.272865 (+ 163us) server_negotiation.cc:299] Negotiation successful
0619 10:17:37.272897 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260619 10:17:37.272979 8824 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:37.270395 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.67:42531 (local address 127.0.206.65:33079)
0619 10:17:37.270582 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0619 10:17:37.270591 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0619 10:17:37.270665 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 10:17:37.270790 (+ 125us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 10:17:37.270790 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 10:17:37.270792 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 10:17:37.270866 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:37.270868 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:37.271448 (+ 580us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:37.271449 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:37.272188 (+ 739us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:37.272190 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:37.272695 (+ 505us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:37.272696 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:37.272768 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:37.272771 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 10:17:37.272845 (+ 74us) client_negotiation.cc:240] Negotiation successful
0619 10:17:37.272887 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":164,"thread_start_us":110,"threads_started":1}
I20260619 10:17:37.273130 8604 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "098e55193002499d98f043abb96af42b" candidate_uuid: "0c4c7226ac8e48ca8d708ea8bf3b76f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eed7c39853c04344aacc55afdd26f1fe" is_pre_election: true
I20260619 10:17:37.273283 8604 raft_consensus.cc:2454] T 098e55193002499d98f043abb96af42b P eed7c39853c04344aacc55afdd26f1fe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0c4c7226ac8e48ca8d708ea8bf3b76f7 in term 0.
I20260619 10:17:37.273535 8398 leader_election.cc:302] T 098e55193002499d98f043abb96af42b P 0c4c7226ac8e48ca8d708ea8bf3b76f7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0c4c7226ac8e48ca8d708ea8bf3b76f7, eed7c39853c04344aacc55afdd26f1fe; no voters:
I20260619 10:17:37.273663 8819 raft_consensus.cc:2781] T 098e55193002499d98f043abb96af42b P 0c4c7226ac8e48ca8d708ea8bf3b76f7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 10:17:37.273645 8744 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "098e55193002499d98f043abb96af42b" candidate_uuid: "0c4c7226ac8e48ca8d708ea8bf3b76f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e4f629e966e419fb21b57161abdf46c" is_pre_election: true
I20260619 10:17:37.273723 8819 raft_consensus.cc:482] T 098e55193002499d98f043abb96af42b P 0c4c7226ac8e48ca8d708ea8bf3b76f7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 10:17:37.273751 8819 raft_consensus.cc:3037] T 098e55193002499d98f043abb96af42b P 0c4c7226ac8e48ca8d708ea8bf3b76f7 [term 0 FOLLOWER]: Advancing to term 1
I20260619 10:17:37.273789 8744 raft_consensus.cc:2454] T 098e55193002499d98f043abb96af42b P 0e4f629e966e419fb21b57161abdf46c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0c4c7226ac8e48ca8d708ea8bf3b76f7 in term 0.
I20260619 10:17:37.274726 8819 raft_consensus.cc:504] T 098e55193002499d98f043abb96af42b P 0c4c7226ac8e48ca8d708ea8bf3b76f7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eed7c39853c04344aacc55afdd26f1fe" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 36277 } } peers { permanent_uuid: "0c4c7226ac8e48ca8d708ea8bf3b76f7" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 37567 } } peers { permanent_uuid: "0e4f629e966e419fb21b57161abdf46c" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 42531 } }
I20260619 10:17:37.274848 8819 leader_election.cc:288] T 098e55193002499d98f043abb96af42b P 0c4c7226ac8e48ca8d708ea8bf3b76f7 [CANDIDATE]: Term 1 election: Requested vote from peers eed7c39853c04344aacc55afdd26f1fe (127.0.206.66:36277), 0e4f629e966e419fb21b57161abdf46c (127.0.206.67:42531)
I20260619 10:17:37.275041 8744 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "098e55193002499d98f043abb96af42b" candidate_uuid: "0c4c7226ac8e48ca8d708ea8bf3b76f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e4f629e966e419fb21b57161abdf46c"
I20260619 10:17:37.275116 8744 raft_consensus.cc:3037] T 098e55193002499d98f043abb96af42b P 0e4f629e966e419fb21b57161abdf46c [term 0 FOLLOWER]: Advancing to term 1
I20260619 10:17:37.275818 8744 raft_consensus.cc:2454] T 098e55193002499d98f043abb96af42b P 0e4f629e966e419fb21b57161abdf46c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0c4c7226ac8e48ca8d708ea8bf3b76f7 in term 1.
I20260619 10:17:37.275956 8604 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "098e55193002499d98f043abb96af42b" candidate_uuid: "0c4c7226ac8e48ca8d708ea8bf3b76f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eed7c39853c04344aacc55afdd26f1fe"
I20260619 10:17:37.276005 8398 leader_election.cc:302] T 098e55193002499d98f043abb96af42b P 0c4c7226ac8e48ca8d708ea8bf3b76f7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0c4c7226ac8e48ca8d708ea8bf3b76f7, 0e4f629e966e419fb21b57161abdf46c; no voters:
I20260619 10:17:37.276043 8604 raft_consensus.cc:3037] T 098e55193002499d98f043abb96af42b P eed7c39853c04344aacc55afdd26f1fe [term 0 FOLLOWER]: Advancing to term 1
I20260619 10:17:37.276118 8819 raft_consensus.cc:2781] T 098e55193002499d98f043abb96af42b P 0c4c7226ac8e48ca8d708ea8bf3b76f7 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 10:17:37.276233 8819 raft_consensus.cc:686] T 098e55193002499d98f043abb96af42b P 0c4c7226ac8e48ca8d708ea8bf3b76f7 [term 1 LEADER]: Becoming Leader. State: Replica: 0c4c7226ac8e48ca8d708ea8bf3b76f7, State: Running, Role: LEADER
I20260619 10:17:37.276346 8819 consensus_queue.cc:237] T 098e55193002499d98f043abb96af42b P 0c4c7226ac8e48ca8d708ea8bf3b76f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eed7c39853c04344aacc55afdd26f1fe" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 36277 } } peers { permanent_uuid: "0c4c7226ac8e48ca8d708ea8bf3b76f7" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 37567 } } peers { permanent_uuid: "0e4f629e966e419fb21b57161abdf46c" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 42531 } }
I20260619 10:17:37.276870 8604 raft_consensus.cc:2454] T 098e55193002499d98f043abb96af42b P eed7c39853c04344aacc55afdd26f1fe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0c4c7226ac8e48ca8d708ea8bf3b76f7 in term 1.
I20260619 10:17:37.277144 8307 catalog_manager.cc:5495] T 098e55193002499d98f043abb96af42b P 0c4c7226ac8e48ca8d708ea8bf3b76f7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0c4c7226ac8e48ca8d708ea8bf3b76f7 (127.0.206.65). New cstate: current_term: 1 leader_uuid: "0c4c7226ac8e48ca8d708ea8bf3b76f7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eed7c39853c04344aacc55afdd26f1fe" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 36277 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0c4c7226ac8e48ca8d708ea8bf3b76f7" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 37567 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0e4f629e966e419fb21b57161abdf46c" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 42531 } health_report { overall_health: UNKNOWN } } }
I20260619 10:17:37.299546 8813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:37.297463 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43060 (local address 127.0.206.65:37567)
0619 10:17:37.297520 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0619 10:17:37.297522 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 10:17:37.297532 (+ 10us) server_negotiation.cc:407] Connection header received
0619 10:17:37.297571 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:17:37.297572 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:17:37.297591 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:17:37.297650 (+ 59us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 10:17:37.297754 (+ 104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:37.298213 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:37.298942 (+ 729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:37.299128 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:37.299181 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:37.299299 (+ 118us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 10:17:37.299390 (+ 91us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 10:17:37.299447 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0619 10:17:37.299476 (+ 29us) server_negotiation.cc:299] Negotiation successful
0619 10:17:37.299492 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260619 10:17:37.301311 8744 raft_consensus.cc:1261] T 098e55193002499d98f043abb96af42b P 0e4f629e966e419fb21b57161abdf46c [term 1 FOLLOWER]: Refusing update from remote peer 0c4c7226ac8e48ca8d708ea8bf3b76f7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 10:17:37.301311 8604 raft_consensus.cc:1261] T 098e55193002499d98f043abb96af42b P eed7c39853c04344aacc55afdd26f1fe [term 1 FOLLOWER]: Refusing update from remote peer 0c4c7226ac8e48ca8d708ea8bf3b76f7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 10:17:37.301654 8819 consensus_queue.cc:1035] T 098e55193002499d98f043abb96af42b P 0c4c7226ac8e48ca8d708ea8bf3b76f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eed7c39853c04344aacc55afdd26f1fe" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 36277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 10:17:37.301779 8825 consensus_queue.cc:1035] T 098e55193002499d98f043abb96af42b P 0c4c7226ac8e48ca8d708ea8bf3b76f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e4f629e966e419fb21b57161abdf46c" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 42531 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260619 10:17:37.303870 8511 tablet.cc:2369] T 098e55193002499d98f043abb96af42b P 0c4c7226ac8e48ca8d708ea8bf3b76f7: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 10:17:37.304006 8828 mvcc.cc:205] Tried to move back new op lower bound from 7298515997903458304 to 7298515997804347392. Current Snapshot: MvccSnapshot[applied={T|T < 7298515997903458304}]
I20260619 10:17:37.304329 8830 mvcc.cc:205] Tried to move back new op lower bound from 7298515997903458304 to 7298515997804347392. Current Snapshot: MvccSnapshot[applied={T|T < 7298515997903458304}]
I20260619 10:17:37.304419 8829 mvcc.cc:205] Tried to move back new op lower bound from 7298515997903458304 to 7298515997804347392. Current Snapshot: MvccSnapshot[applied={T|T < 7298515997903458304}]
I20260619 10:17:37.306694 8307 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:58402:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 10:17:37.306792 8307 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:58402:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 10:17:37.307502 8307 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f99ceb70a6684009b753712e007c5de5: Sending DeleteTablet for 3 replicas of tablet 098e55193002499d98f043abb96af42b
I20260619 10:17:37.307838 8444 tablet_service.cc:1514] Processing DeleteTablet for tablet 098e55193002499d98f043abb96af42b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 10:17:37 UTC) from {username='oryx', principal='oryx/127.0.206.126@KRBTEST.COM'} at 127.0.0.1:43058
I20260619 10:17:37.307968 8584 tablet_service.cc:1514] Processing DeleteTablet for tablet 098e55193002499d98f043abb96af42b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 10:17:37 UTC) from {username='oryx', principal='oryx/127.0.206.126@KRBTEST.COM'} at 127.0.0.1:36298
I20260619 10:17:37.308018 8724 tablet_service.cc:1514] Processing DeleteTablet for tablet 098e55193002499d98f043abb96af42b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 10:17:37 UTC) from {username='oryx', principal='oryx/127.0.206.126@KRBTEST.COM'} at 127.0.0.1:36834
I20260619 10:17:37.308049 8307 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:58402:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260619 10:17:37.308218 8838 tablet_replica.cc:330] T 098e55193002499d98f043abb96af42b P 0c4c7226ac8e48ca8d708ea8bf3b76f7: stopping tablet replica
W20260619 10:17:37.308220 8307 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 10:17:37.308302 8838 raft_consensus.cc:2229] T 098e55193002499d98f043abb96af42b P 0c4c7226ac8e48ca8d708ea8bf3b76f7 [term 1 LEADER]: Raft consensus shutting down.
I20260619 10:17:37.308538 8838 raft_consensus.cc:2258] T 098e55193002499d98f043abb96af42b P 0c4c7226ac8e48ca8d708ea8bf3b76f7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 10:17:37.308980 8838 ts_tablet_manager.cc:1905] T 098e55193002499d98f043abb96af42b P 0c4c7226ac8e48ca8d708ea8bf3b76f7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 10:17:37.309365 8444 tablet_service.cc:1467] Processing CreateTablet for tablet dc0c7f2b3661431896754d048cba6f5a (DEFAULT_TABLE table=test-table [id=7940c309a7d64070b673aa22a36cfb11]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 10:17:37.309398 8724 tablet_service.cc:1467] Processing CreateTablet for tablet dc0c7f2b3661431896754d048cba6f5a (DEFAULT_TABLE table=test-table [id=7940c309a7d64070b673aa22a36cfb11]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 10:17:37.309465 8444 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dc0c7f2b3661431896754d048cba6f5a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 10:17:37.309478 8584 tablet_service.cc:1467] Processing CreateTablet for tablet dc0c7f2b3661431896754d048cba6f5a (DEFAULT_TABLE table=test-table [id=7940c309a7d64070b673aa22a36cfb11]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 10:17:37.309511 8724 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dc0c7f2b3661431896754d048cba6f5a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 10:17:37.309573 8584 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dc0c7f2b3661431896754d048cba6f5a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 10:17:37.310745 8815 tablet_bootstrap.cc:492] T dc0c7f2b3661431896754d048cba6f5a P 0c4c7226ac8e48ca8d708ea8bf3b76f7: Bootstrap starting.
I20260619 10:17:37.311020 8820 tablet_bootstrap.cc:492] T dc0c7f2b3661431896754d048cba6f5a P eed7c39853c04344aacc55afdd26f1fe: Bootstrap starting.
I20260619 10:17:37.311022 8816 tablet_bootstrap.cc:492] T dc0c7f2b3661431896754d048cba6f5a P 0e4f629e966e419fb21b57161abdf46c: Bootstrap starting.
I20260619 10:17:37.311129 8815 tablet_bootstrap.cc:654] T dc0c7f2b3661431896754d048cba6f5a P 0c4c7226ac8e48ca8d708ea8bf3b76f7: Neither blocks nor log segments found. Creating new log.
I20260619 10:17:37.311384 8820 tablet_bootstrap.cc:654] T dc0c7f2b3661431896754d048cba6f5a P eed7c39853c04344aacc55afdd26f1fe: Neither blocks nor log segments found. Creating new log.
I20260619 10:17:37.311437 8816 tablet_bootstrap.cc:654] T dc0c7f2b3661431896754d048cba6f5a P 0e4f629e966e419fb21b57161abdf46c: Neither blocks nor log segments found. Creating new log.
I20260619 10:17:37.311872 8815 tablet_bootstrap.cc:492] T dc0c7f2b3661431896754d048cba6f5a P 0c4c7226ac8e48ca8d708ea8bf3b76f7: No bootstrap required, opened a new log
I20260619 10:17:37.311934 8815 ts_tablet_manager.cc:1397] T dc0c7f2b3661431896754d048cba6f5a P 0c4c7226ac8e48ca8d708ea8bf3b76f7: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260619 10:17:37.312090 8815 raft_consensus.cc:348] T dc0c7f2b3661431896754d048cba6f5a P 0c4c7226ac8e48ca8d708ea8bf3b76f7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c4c7226ac8e48ca8d708ea8bf3b76f7" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 37567 } } peers { permanent_uuid: "eed7c39853c04344aacc55afdd26f1fe" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 36277 } } peers { permanent_uuid: "0e4f629e966e419fb21b57161abdf46c" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 42531 } }
I20260619 10:17:37.312153 8815 raft_consensus.cc:374] T dc0c7f2b3661431896754d048cba6f5a P 0c4c7226ac8e48ca8d708ea8bf3b76f7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 10:17:37.312171 8815 raft_consensus.cc:729] T dc0c7f2b3661431896754d048cba6f5a P 0c4c7226ac8e48ca8d708ea8bf3b76f7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0c4c7226ac8e48ca8d708ea8bf3b76f7, State: Initialized, Role: FOLLOWER
I20260619 10:17:37.312233 8815 consensus_queue.cc:260] T dc0c7f2b3661431896754d048cba6f5a P 0c4c7226ac8e48ca8d708ea8bf3b76f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c4c7226ac8e48ca8d708ea8bf3b76f7" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 37567 } } peers { permanent_uuid: "eed7c39853c04344aacc55afdd26f1fe" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 36277 } } peers { permanent_uuid: "0e4f629e966e419fb21b57161abdf46c" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 42531 } }
I20260619 10:17:37.312319 8815 ts_tablet_manager.cc:1428] T dc0c7f2b3661431896754d048cba6f5a P 0c4c7226ac8e48ca8d708ea8bf3b76f7: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260619 10:17:37.312938 8838 ts_tablet_manager.cc:1918] T 098e55193002499d98f043abb96af42b P 0c4c7226ac8e48ca8d708ea8bf3b76f7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 10:17:37.313007 8838 log.cc:1196] T 098e55193002499d98f043abb96af42b P 0c4c7226ac8e48ca8d708ea8bf3b76f7: Deleting WAL directory at /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781864224577058-825-0/minicluster-data/ts-0/wal/wals/098e55193002499d98f043abb96af42b
I20260619 10:17:37.313274 8838 ts_tablet_manager.cc:1939] T 098e55193002499d98f043abb96af42b P 0c4c7226ac8e48ca8d708ea8bf3b76f7: Deleting consensus metadata
I20260619 10:17:37.313637 8293 catalog_manager.cc:4841] TS 0c4c7226ac8e48ca8d708ea8bf3b76f7 (127.0.206.65:37567): tablet 098e55193002499d98f043abb96af42b (table test-table [id=3094b700c8f84a6195757f172bb1466c]) successfully deleted
I20260619 10:17:37.314988 8820 tablet_bootstrap.cc:492] T dc0c7f2b3661431896754d048cba6f5a P eed7c39853c04344aacc55afdd26f1fe: No bootstrap required, opened a new log
I20260619 10:17:37.315047 8820 ts_tablet_manager.cc:1397] T dc0c7f2b3661431896754d048cba6f5a P eed7c39853c04344aacc55afdd26f1fe: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20260619 10:17:37.315165 8840 tablet_replica.cc:330] T 098e55193002499d98f043abb96af42b P 0e4f629e966e419fb21b57161abdf46c: stopping tablet replica
I20260619 10:17:37.315253 8839 tablet_replica.cc:330] T 098e55193002499d98f043abb96af42b P eed7c39853c04344aacc55afdd26f1fe: stopping tablet replica
I20260619 10:17:37.315210 8820 raft_consensus.cc:348] T dc0c7f2b3661431896754d048cba6f5a P eed7c39853c04344aacc55afdd26f1fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c4c7226ac8e48ca8d708ea8bf3b76f7" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 37567 } } peers { permanent_uuid: "eed7c39853c04344aacc55afdd26f1fe" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 36277 } } peers { permanent_uuid: "0e4f629e966e419fb21b57161abdf46c" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 42531 } }
I20260619 10:17:37.315294 8820 raft_consensus.cc:374] T dc0c7f2b3661431896754d048cba6f5a P eed7c39853c04344aacc55afdd26f1fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 10:17:37.315325 8820 raft_consensus.cc:729] T dc0c7f2b3661431896754d048cba6f5a P eed7c39853c04344aacc55afdd26f1fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eed7c39853c04344aacc55afdd26f1fe, State: Initialized, Role: FOLLOWER
I20260619 10:17:37.315348 8839 raft_consensus.cc:2229] T 098e55193002499d98f043abb96af42b P eed7c39853c04344aacc55afdd26f1fe [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 10:17:37.315371 8820 consensus_queue.cc:260] T dc0c7f2b3661431896754d048cba6f5a P eed7c39853c04344aacc55afdd26f1fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c4c7226ac8e48ca8d708ea8bf3b76f7" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 37567 } } peers { permanent_uuid: "eed7c39853c04344aacc55afdd26f1fe" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 36277 } } peers { permanent_uuid: "0e4f629e966e419fb21b57161abdf46c" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 42531 } }
I20260619 10:17:37.315447 8839 raft_consensus.cc:2258] T 098e55193002499d98f043abb96af42b P eed7c39853c04344aacc55afdd26f1fe [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 10:17:37.315460 8820 ts_tablet_manager.cc:1428] T dc0c7f2b3661431896754d048cba6f5a P eed7c39853c04344aacc55afdd26f1fe: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260619 10:17:37.315608 8840 raft_consensus.cc:2229] T 098e55193002499d98f043abb96af42b P 0e4f629e966e419fb21b57161abdf46c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 10:17:37.315173 8816 tablet_bootstrap.cc:492] T dc0c7f2b3661431896754d048cba6f5a P 0e4f629e966e419fb21b57161abdf46c: No bootstrap required, opened a new log
I20260619 10:17:37.315703 8816 ts_tablet_manager.cc:1397] T dc0c7f2b3661431896754d048cba6f5a P 0e4f629e966e419fb21b57161abdf46c: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.000s
I20260619 10:17:37.315742 8840 raft_consensus.cc:2258] T 098e55193002499d98f043abb96af42b P 0e4f629e966e419fb21b57161abdf46c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 10:17:37.315876 8816 raft_consensus.cc:348] T dc0c7f2b3661431896754d048cba6f5a P 0e4f629e966e419fb21b57161abdf46c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c4c7226ac8e48ca8d708ea8bf3b76f7" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 37567 } } peers { permanent_uuid: "eed7c39853c04344aacc55afdd26f1fe" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 36277 } } peers { permanent_uuid: "0e4f629e966e419fb21b57161abdf46c" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 42531 } }
I20260619 10:17:37.315951 8816 raft_consensus.cc:374] T dc0c7f2b3661431896754d048cba6f5a P 0e4f629e966e419fb21b57161abdf46c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 10:17:37.315972 8816 raft_consensus.cc:729] T dc0c7f2b3661431896754d048cba6f5a P 0e4f629e966e419fb21b57161abdf46c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e4f629e966e419fb21b57161abdf46c, State: Initialized, Role: FOLLOWER
I20260619 10:17:37.316012 8816 consensus_queue.cc:260] T dc0c7f2b3661431896754d048cba6f5a P 0e4f629e966e419fb21b57161abdf46c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c4c7226ac8e48ca8d708ea8bf3b76f7" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 37567 } } peers { permanent_uuid: "eed7c39853c04344aacc55afdd26f1fe" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 36277 } } peers { permanent_uuid: "0e4f629e966e419fb21b57161abdf46c" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 42531 } }
I20260619 10:17:37.316102 8816 ts_tablet_manager.cc:1428] T dc0c7f2b3661431896754d048cba6f5a P 0e4f629e966e419fb21b57161abdf46c: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260619 10:17:37.316183 8839 ts_tablet_manager.cc:1905] T 098e55193002499d98f043abb96af42b P eed7c39853c04344aacc55afdd26f1fe: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 10:17:37.317015 8840 ts_tablet_manager.cc:1905] T 098e55193002499d98f043abb96af42b P 0e4f629e966e419fb21b57161abdf46c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 10:17:37.317919 8839 ts_tablet_manager.cc:1918] T 098e55193002499d98f043abb96af42b P eed7c39853c04344aacc55afdd26f1fe: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 10:17:37.318004 8839 log.cc:1196] T 098e55193002499d98f043abb96af42b P eed7c39853c04344aacc55afdd26f1fe: Deleting WAL directory at /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781864224577058-825-0/minicluster-data/ts-1/wal/wals/098e55193002499d98f043abb96af42b
I20260619 10:17:37.318288 8839 ts_tablet_manager.cc:1939] T 098e55193002499d98f043abb96af42b P eed7c39853c04344aacc55afdd26f1fe: Deleting consensus metadata
I20260619 10:17:37.318464 8840 ts_tablet_manager.cc:1918] T 098e55193002499d98f043abb96af42b P 0e4f629e966e419fb21b57161abdf46c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 10:17:37.318542 8840 log.cc:1196] T 098e55193002499d98f043abb96af42b P 0e4f629e966e419fb21b57161abdf46c: Deleting WAL directory at /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781864224577058-825-0/minicluster-data/ts-2/wal/wals/098e55193002499d98f043abb96af42b
I20260619 10:17:37.318678 8294 catalog_manager.cc:4841] TS eed7c39853c04344aacc55afdd26f1fe (127.0.206.66:36277): tablet 098e55193002499d98f043abb96af42b (table test-table [id=3094b700c8f84a6195757f172bb1466c]) successfully deleted
I20260619 10:17:37.318794 8840 ts_tablet_manager.cc:1939] T 098e55193002499d98f043abb96af42b P 0e4f629e966e419fb21b57161abdf46c: Deleting consensus metadata
I20260619 10:17:37.319140 8294 catalog_manager.cc:4841] TS 0e4f629e966e419fb21b57161abdf46c (127.0.206.67:42531): tablet 098e55193002499d98f043abb96af42b (table test-table [id=3094b700c8f84a6195757f172bb1466c]) successfully deleted
W20260619 10:17:37.359584 8791 tablet.cc:2369] T dc0c7f2b3661431896754d048cba6f5a P 0e4f629e966e419fb21b57161abdf46c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 10:17:37.406967 8823 raft_consensus.cc:482] T dc0c7f2b3661431896754d048cba6f5a P eed7c39853c04344aacc55afdd26f1fe [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 10:17:37.407069 8823 raft_consensus.cc:504] T dc0c7f2b3661431896754d048cba6f5a P eed7c39853c04344aacc55afdd26f1fe [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c4c7226ac8e48ca8d708ea8bf3b76f7" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 37567 } } peers { permanent_uuid: "eed7c39853c04344aacc55afdd26f1fe" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 36277 } } peers { permanent_uuid: "0e4f629e966e419fb21b57161abdf46c" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 42531 } }
I20260619 10:17:37.407441 8823 leader_election.cc:288] T dc0c7f2b3661431896754d048cba6f5a P eed7c39853c04344aacc55afdd26f1fe [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0c4c7226ac8e48ca8d708ea8bf3b76f7 (127.0.206.65:37567), 0e4f629e966e419fb21b57161abdf46c (127.0.206.67:42531)
I20260619 10:17:37.410897 8813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:37.407636 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.66:41089 (local address 127.0.206.65:37567)
0619 10:17:37.407677 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0619 10:17:37.407678 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 10:17:37.407687 (+ 9us) server_negotiation.cc:407] Connection header received
0619 10:17:37.407782 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:17:37.407783 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:17:37.407794 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:17:37.407840 (+ 46us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 10:17:37.408060 (+ 220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:37.408540 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:37.409904 (+ 1364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:37.410494 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:37.410583 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:37.410594 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0619 10:17:37.410781 (+ 187us) server_negotiation.cc:299] Negotiation successful
0619 10:17:37.410824 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260619 10:17:37.410897 8543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:37.407577 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.65:37567 (local address 127.0.206.66:41089)
0619 10:17:37.407643 (+ 66us) negotiation.cc:107] Waiting for socket to connect
0619 10:17:37.407656 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 10:17:37.407699 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 10:17:37.407886 (+ 187us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 10:17:37.407886 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 10:17:37.407888 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 10:17:37.407975 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:37.407978 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:37.408637 (+ 659us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:37.408638 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:37.409796 (+ 1158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:37.409799 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:37.410582 (+ 783us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:37.410590 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:37.410715 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:37.410719 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 10:17:37.410774 (+ 55us) client_negotiation.cc:240] Negotiation successful
0619 10:17:37.410833 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":50,"mutex_wait_us":14}
I20260619 10:17:37.411057 8814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:37.407797 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.66:33045 (local address 127.0.206.67:42531)
0619 10:17:37.407845 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0619 10:17:37.407847 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 10:17:37.407857 (+ 10us) server_negotiation.cc:407] Connection header received
0619 10:17:37.407887 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:17:37.407888 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:17:37.407898 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:17:37.407967 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 10:17:37.408131 (+ 164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:37.408892 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:37.410229 (+ 1337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:37.410646 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:37.410717 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:37.410725 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0619 10:17:37.410920 (+ 195us) server_negotiation.cc:299] Negotiation successful
0619 10:17:37.410980 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260619 10:17:37.411216 8464 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dc0c7f2b3661431896754d048cba6f5a" candidate_uuid: "eed7c39853c04344aacc55afdd26f1fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0c4c7226ac8e48ca8d708ea8bf3b76f7" is_pre_election: true
I20260619 10:17:37.411312 8844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:37.407617 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.67:42531 (local address 127.0.206.66:33045)
0619 10:17:37.407731 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0619 10:17:37.407740 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0619 10:17:37.407783 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 10:17:37.407977 (+ 194us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 10:17:37.407978 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 10:17:37.407980 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 10:17:37.408048 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:37.408050 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:37.409118 (+ 1068us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:37.409120 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:37.410150 (+ 1030us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:37.410152 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:37.410722 (+ 570us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:37.410723 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:37.410807 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:37.410811 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 10:17:37.411182 (+ 371us) client_negotiation.cc:240] Negotiation successful
0619 10:17:37.411231 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":75,"threads_started":1}
I20260619 10:17:37.411325 8464 raft_consensus.cc:2454] T dc0c7f2b3661431896754d048cba6f5a P 0c4c7226ac8e48ca8d708ea8bf3b76f7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eed7c39853c04344aacc55afdd26f1fe in term 0.
I20260619 10:17:37.411514 8537 leader_election.cc:302] T dc0c7f2b3661431896754d048cba6f5a P eed7c39853c04344aacc55afdd26f1fe [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0c4c7226ac8e48ca8d708ea8bf3b76f7, eed7c39853c04344aacc55afdd26f1fe; no voters:
I20260619 10:17:37.411548 8744 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dc0c7f2b3661431896754d048cba6f5a" candidate_uuid: "eed7c39853c04344aacc55afdd26f1fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e4f629e966e419fb21b57161abdf46c" is_pre_election: true
I20260619 10:17:37.411633 8823 raft_consensus.cc:2781] T dc0c7f2b3661431896754d048cba6f5a P eed7c39853c04344aacc55afdd26f1fe [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 10:17:37.411643 8744 raft_consensus.cc:2454] T dc0c7f2b3661431896754d048cba6f5a P 0e4f629e966e419fb21b57161abdf46c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eed7c39853c04344aacc55afdd26f1fe in term 0.
I20260619 10:17:37.411667 8823 raft_consensus.cc:482] T dc0c7f2b3661431896754d048cba6f5a P eed7c39853c04344aacc55afdd26f1fe [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 10:17:37.411691 8823 raft_consensus.cc:3037] T dc0c7f2b3661431896754d048cba6f5a P eed7c39853c04344aacc55afdd26f1fe [term 0 FOLLOWER]: Advancing to term 1
I20260619 10:17:37.412499 8823 raft_consensus.cc:504] T dc0c7f2b3661431896754d048cba6f5a P eed7c39853c04344aacc55afdd26f1fe [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c4c7226ac8e48ca8d708ea8bf3b76f7" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 37567 } } peers { permanent_uuid: "eed7c39853c04344aacc55afdd26f1fe" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 36277 } } peers { permanent_uuid: "0e4f629e966e419fb21b57161abdf46c" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 42531 } }
I20260619 10:17:37.412642 8823 leader_election.cc:288] T dc0c7f2b3661431896754d048cba6f5a P eed7c39853c04344aacc55afdd26f1fe [CANDIDATE]: Term 1 election: Requested vote from peers 0c4c7226ac8e48ca8d708ea8bf3b76f7 (127.0.206.65:37567), 0e4f629e966e419fb21b57161abdf46c (127.0.206.67:42531)
I20260619 10:17:37.412844 8744 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dc0c7f2b3661431896754d048cba6f5a" candidate_uuid: "eed7c39853c04344aacc55afdd26f1fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e4f629e966e419fb21b57161abdf46c"
I20260619 10:17:37.412936 8744 raft_consensus.cc:3037] T dc0c7f2b3661431896754d048cba6f5a P 0e4f629e966e419fb21b57161abdf46c [term 0 FOLLOWER]: Advancing to term 1
I20260619 10:17:37.412902 8464 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dc0c7f2b3661431896754d048cba6f5a" candidate_uuid: "eed7c39853c04344aacc55afdd26f1fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0c4c7226ac8e48ca8d708ea8bf3b76f7"
I20260619 10:17:37.412995 8464 raft_consensus.cc:3037] T dc0c7f2b3661431896754d048cba6f5a P 0c4c7226ac8e48ca8d708ea8bf3b76f7 [term 0 FOLLOWER]: Advancing to term 1
I20260619 10:17:37.413715 8464 raft_consensus.cc:2454] T dc0c7f2b3661431896754d048cba6f5a P 0c4c7226ac8e48ca8d708ea8bf3b76f7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eed7c39853c04344aacc55afdd26f1fe in term 1.
I20260619 10:17:37.413753 8744 raft_consensus.cc:2454] T dc0c7f2b3661431896754d048cba6f5a P 0e4f629e966e419fb21b57161abdf46c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eed7c39853c04344aacc55afdd26f1fe in term 1.
I20260619 10:17:37.413897 8537 leader_election.cc:302] T dc0c7f2b3661431896754d048cba6f5a P eed7c39853c04344aacc55afdd26f1fe [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0c4c7226ac8e48ca8d708ea8bf3b76f7, eed7c39853c04344aacc55afdd26f1fe; no voters:
I20260619 10:17:37.413985 8823 raft_consensus.cc:2781] T dc0c7f2b3661431896754d048cba6f5a P eed7c39853c04344aacc55afdd26f1fe [term 1 FOLLOWER]: Leader election won for term 1
I20260619 10:17:37.414100 8823 raft_consensus.cc:686] T dc0c7f2b3661431896754d048cba6f5a P eed7c39853c04344aacc55afdd26f1fe [term 1 LEADER]: Becoming Leader. State: Replica: eed7c39853c04344aacc55afdd26f1fe, State: Running, Role: LEADER
I20260619 10:17:37.414170 8823 consensus_queue.cc:237] T dc0c7f2b3661431896754d048cba6f5a P eed7c39853c04344aacc55afdd26f1fe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c4c7226ac8e48ca8d708ea8bf3b76f7" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 37567 } } peers { permanent_uuid: "eed7c39853c04344aacc55afdd26f1fe" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 36277 } } peers { permanent_uuid: "0e4f629e966e419fb21b57161abdf46c" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 42531 } }
I20260619 10:17:37.414904 8307 catalog_manager.cc:5495] T dc0c7f2b3661431896754d048cba6f5a P eed7c39853c04344aacc55afdd26f1fe reported cstate change: term changed from 0 to 1, leader changed from <none> to eed7c39853c04344aacc55afdd26f1fe (127.0.206.66). New cstate: current_term: 1 leader_uuid: "eed7c39853c04344aacc55afdd26f1fe" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c4c7226ac8e48ca8d708ea8bf3b76f7" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 37567 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eed7c39853c04344aacc55afdd26f1fe" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 36277 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0e4f629e966e419fb21b57161abdf46c" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 42531 } health_report { overall_health: UNKNOWN } } }
Jun 19 10:17:37 dist-test-slave-6pt2 krb5kdc[8262](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864256, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.206.126@KRBTEST.COM for oryx/127.0.206.126@KRBTEST.COM
I20260619 10:17:37.465561 8852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:37.459255 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:45657 (local address 127.0.0.1:58414)
0619 10:17:37.459415 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0619 10:17:37.459425 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 10:17:37.459490 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 10:17:37.459613 (+ 123us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 10:17:37.459614 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 10:17:37.459905 (+ 291us) client_negotiation.cc:189] Negotiated authn=SASL
0619 10:17:37.459951 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:37.459953 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:37.460553 (+ 600us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:37.460554 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:37.461080 (+ 526us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:37.461082 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:37.461366 (+ 284us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:37.461367 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:37.461403 (+ 36us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:37.461980 (+ 577us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 10:17:37.461985 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 10:17:37.463367 (+ 1382us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 10:17:37.464713 (+ 1346us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 10:17:37.464718 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 10:17:37.464720 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 10:17:37.464902 (+ 182us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 10:17:37.465057 (+ 155us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 10:17:37.465057 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 10:17:37.465058 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 10:17:37.465108 (+ 50us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 10:17:37.465290 (+ 182us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 10:17:37.465295 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 10:17:37.465333 (+ 38us) client_negotiation.cc:769] Sending connection context
0619 10:17:37.465385 (+ 52us) client_negotiation.cc:240] Negotiation successful
0619 10:17:37.465470 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":93,"threads_started":1}
I20260619 10:17:37.465642 8356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:37.459288 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58414 (local address 127.0.206.126:45657)
0619 10:17:37.459348 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0619 10:17:37.459351 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 10:17:37.459486 (+ 135us) server_negotiation.cc:407] Connection header received
0619 10:17:37.459535 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:17:37.459535 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:17:37.459544 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:17:37.459601 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:17:37.460015 (+ 414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:37.460490 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:37.461143 (+ 653us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:37.461310 (+ 167us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:37.461359 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:37.463457 (+ 2098us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 10:17:37.463458 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 10:17:37.463458 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 10:17:37.463464 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 10:17:37.464649 (+ 1185us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:17:37.464961 (+ 312us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:17:37.464962 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:17:37.464962 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:17:37.465008 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:17:37.465155 (+ 147us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:17:37.465156 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:17:37.465156 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:17:37.465245 (+ 89us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 10:17:37.465291 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0619 10:17:37.465422 (+ 131us) server_negotiation.cc:299] Negotiation successful
0619 10:17:37.465512 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":34,"server-negotiator.queue_time_us":34}
I20260619 10:17:37.467365 8307 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.0.206.126@KRBTEST.COM'} at 127.0.0.1:58414:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260619 10:17:37.467578 8307 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 10:17:37.468969 8444 tablet_service.cc:1467] Processing CreateTablet for tablet a2ab22162c954be09c0370e92d9369e5 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=d425d5a60d1246cfa07c5f761f978792]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260619 10:17:37.468998 8724 tablet_service.cc:1467] Processing CreateTablet for tablet a2ab22162c954be09c0370e92d9369e5 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=d425d5a60d1246cfa07c5f761f978792]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260619 10:17:37.469137 8444 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a2ab22162c954be09c0370e92d9369e5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 10:17:37.469144 8724 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a2ab22162c954be09c0370e92d9369e5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 10:17:37.470214 8815 tablet_bootstrap.cc:492] T a2ab22162c954be09c0370e92d9369e5 P 0c4c7226ac8e48ca8d708ea8bf3b76f7: Bootstrap starting.
I20260619 10:17:37.470288 8816 tablet_bootstrap.cc:492] T a2ab22162c954be09c0370e92d9369e5 P 0e4f629e966e419fb21b57161abdf46c: Bootstrap starting.
I20260619 10:17:37.470314 8584 tablet_service.cc:1467] Processing CreateTablet for tablet a2ab22162c954be09c0370e92d9369e5 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=d425d5a60d1246cfa07c5f761f978792]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260619 10:17:37.470432 8584 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a2ab22162c954be09c0370e92d9369e5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 10:17:37.470535 8815 tablet_bootstrap.cc:654] T a2ab22162c954be09c0370e92d9369e5 P 0c4c7226ac8e48ca8d708ea8bf3b76f7: Neither blocks nor log segments found. Creating new log.
I20260619 10:17:37.470701 8816 tablet_bootstrap.cc:654] T a2ab22162c954be09c0370e92d9369e5 P 0e4f629e966e419fb21b57161abdf46c: Neither blocks nor log segments found. Creating new log.
I20260619 10:17:37.471115 8815 tablet_bootstrap.cc:492] T a2ab22162c954be09c0370e92d9369e5 P 0c4c7226ac8e48ca8d708ea8bf3b76f7: No bootstrap required, opened a new log
I20260619 10:17:37.471172 8815 ts_tablet_manager.cc:1397] T a2ab22162c954be09c0370e92d9369e5 P 0c4c7226ac8e48ca8d708ea8bf3b76f7: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260619 10:17:37.471282 8815 raft_consensus.cc:348] T a2ab22162c954be09c0370e92d9369e5 P 0c4c7226ac8e48ca8d708ea8bf3b76f7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c4c7226ac8e48ca8d708ea8bf3b76f7" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 37567 } } peers { permanent_uuid: "eed7c39853c04344aacc55afdd26f1fe" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 36277 } } peers { permanent_uuid: "0e4f629e966e419fb21b57161abdf46c" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 42531 } }
I20260619 10:17:37.471323 8816 tablet_bootstrap.cc:492] T a2ab22162c954be09c0370e92d9369e5 P 0e4f629e966e419fb21b57161abdf46c: No bootstrap required, opened a new log
I20260619 10:17:37.471354 8815 raft_consensus.cc:374] T a2ab22162c954be09c0370e92d9369e5 P 0c4c7226ac8e48ca8d708ea8bf3b76f7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 10:17:37.471364 8816 ts_tablet_manager.cc:1397] T a2ab22162c954be09c0370e92d9369e5 P 0e4f629e966e419fb21b57161abdf46c: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260619 10:17:37.471375 8815 raft_consensus.cc:729] T a2ab22162c954be09c0370e92d9369e5 P 0c4c7226ac8e48ca8d708ea8bf3b76f7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0c4c7226ac8e48ca8d708ea8bf3b76f7, State: Initialized, Role: FOLLOWER
I20260619 10:17:37.471416 8815 consensus_queue.cc:260] T a2ab22162c954be09c0370e92d9369e5 P 0c4c7226ac8e48ca8d708ea8bf3b76f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c4c7226ac8e48ca8d708ea8bf3b76f7" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 37567 } } peers { permanent_uuid: "eed7c39853c04344aacc55afdd26f1fe" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 36277 } } peers { permanent_uuid: "0e4f629e966e419fb21b57161abdf46c" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 42531 } }
I20260619 10:17:37.471505 8815 ts_tablet_manager.cc:1428] T a2ab22162c954be09c0370e92d9369e5 P 0c4c7226ac8e48ca8d708ea8bf3b76f7: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260619 10:17:37.471503 8816 raft_consensus.cc:348] T a2ab22162c954be09c0370e92d9369e5 P 0e4f629e966e419fb21b57161abdf46c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c4c7226ac8e48ca8d708ea8bf3b76f7" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 37567 } } peers { permanent_uuid: "eed7c39853c04344aacc55afdd26f1fe" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 36277 } } peers { permanent_uuid: "0e4f629e966e419fb21b57161abdf46c" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 42531 } }
I20260619 10:17:37.471563 8816 raft_consensus.cc:374] T a2ab22162c954be09c0370e92d9369e5 P 0e4f629e966e419fb21b57161abdf46c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 10:17:37.471589 8816 raft_consensus.cc:729] T a2ab22162c954be09c0370e92d9369e5 P 0e4f629e966e419fb21b57161abdf46c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e4f629e966e419fb21b57161abdf46c, State: Initialized, Role: FOLLOWER
I20260619 10:17:37.471604 8819 tablet_replica.cc:439] T a2ab22162c954be09c0370e92d9369e5 P 0c4c7226ac8e48ca8d708ea8bf3b76f7: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c4c7226ac8e48ca8d708ea8bf3b76f7" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 37567 } } peers { permanent_uuid: "eed7c39853c04344aacc55afdd26f1fe" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 36277 } } peers { permanent_uuid: "0e4f629e966e419fb21b57161abdf46c" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 42531 } } }
I20260619 10:17:37.471630 8816 consensus_queue.cc:260] T a2ab22162c954be09c0370e92d9369e5 P 0e4f629e966e419fb21b57161abdf46c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c4c7226ac8e48ca8d708ea8bf3b76f7" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 37567 } } peers { permanent_uuid: "eed7c39853c04344aacc55afdd26f1fe" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 36277 } } peers { permanent_uuid: "0e4f629e966e419fb21b57161abdf46c" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 42531 } }
I20260619 10:17:37.471673 8819 tablet_replica.cc:442] T a2ab22162c954be09c0370e92d9369e5 P 0c4c7226ac8e48ca8d708ea8bf3b76f7: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 10:17:37.471707 8816 ts_tablet_manager.cc:1428] T a2ab22162c954be09c0370e92d9369e5 P 0e4f629e966e419fb21b57161abdf46c: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260619 10:17:37.471827 8821 tablet_replica.cc:439] T a2ab22162c954be09c0370e92d9369e5 P 0e4f629e966e419fb21b57161abdf46c: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c4c7226ac8e48ca8d708ea8bf3b76f7" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 37567 } } peers { permanent_uuid: "eed7c39853c04344aacc55afdd26f1fe" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 36277 } } peers { permanent_uuid: "0e4f629e966e419fb21b57161abdf46c" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 42531 } } }
I20260619 10:17:37.471935 8821 tablet_replica.cc:442] T a2ab22162c954be09c0370e92d9369e5 P 0e4f629e966e419fb21b57161abdf46c: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 10:17:37.471962 8820 tablet_bootstrap.cc:492] T a2ab22162c954be09c0370e92d9369e5 P eed7c39853c04344aacc55afdd26f1fe: Bootstrap starting.
I20260619 10:17:37.472349 8820 tablet_bootstrap.cc:654] T a2ab22162c954be09c0370e92d9369e5 P eed7c39853c04344aacc55afdd26f1fe: Neither blocks nor log segments found. Creating new log.
I20260619 10:17:37.472965 8820 tablet_bootstrap.cc:492] T a2ab22162c954be09c0370e92d9369e5 P eed7c39853c04344aacc55afdd26f1fe: No bootstrap required, opened a new log
I20260619 10:17:37.473011 8820 ts_tablet_manager.cc:1397] T a2ab22162c954be09c0370e92d9369e5 P eed7c39853c04344aacc55afdd26f1fe: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260619 10:17:37.473152 8820 raft_consensus.cc:348] T a2ab22162c954be09c0370e92d9369e5 P eed7c39853c04344aacc55afdd26f1fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c4c7226ac8e48ca8d708ea8bf3b76f7" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 37567 } } peers { permanent_uuid: "eed7c39853c04344aacc55afdd26f1fe" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 36277 } } peers { permanent_uuid: "0e4f629e966e419fb21b57161abdf46c" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 42531 } }
I20260619 10:17:37.473206 8820 raft_consensus.cc:374] T a2ab22162c954be09c0370e92d9369e5 P eed7c39853c04344aacc55afdd26f1fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 10:17:37.473219 8820 raft_consensus.cc:729] T a2ab22162c954be09c0370e92d9369e5 P eed7c39853c04344aacc55afdd26f1fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eed7c39853c04344aacc55afdd26f1fe, State: Initialized, Role: FOLLOWER
I20260619 10:17:37.473281 8820 consensus_queue.cc:260] T a2ab22162c954be09c0370e92d9369e5 P eed7c39853c04344aacc55afdd26f1fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c4c7226ac8e48ca8d708ea8bf3b76f7" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 37567 } } peers { permanent_uuid: "eed7c39853c04344aacc55afdd26f1fe" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 36277 } } peers { permanent_uuid: "0e4f629e966e419fb21b57161abdf46c" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 42531 } }
I20260619 10:17:37.473369 8820 ts_tablet_manager.cc:1428] T a2ab22162c954be09c0370e92d9369e5 P eed7c39853c04344aacc55afdd26f1fe: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260619 10:17:37.473412 8823 tablet_replica.cc:439] T a2ab22162c954be09c0370e92d9369e5 P eed7c39853c04344aacc55afdd26f1fe: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c4c7226ac8e48ca8d708ea8bf3b76f7" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 37567 } } peers { permanent_uuid: "eed7c39853c04344aacc55afdd26f1fe" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 36277 } } peers { permanent_uuid: "0e4f629e966e419fb21b57161abdf46c" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 42531 } } }
I20260619 10:17:37.473479 8823 tablet_replica.cc:442] T a2ab22162c954be09c0370e92d9369e5 P eed7c39853c04344aacc55afdd26f1fe: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 10:17:37.677911 8819 raft_consensus.cc:482] T a2ab22162c954be09c0370e92d9369e5 P 0c4c7226ac8e48ca8d708ea8bf3b76f7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 10:17:37.678023 8819 raft_consensus.cc:504] T a2ab22162c954be09c0370e92d9369e5 P 0c4c7226ac8e48ca8d708ea8bf3b76f7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c4c7226ac8e48ca8d708ea8bf3b76f7" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 37567 } } peers { permanent_uuid: "eed7c39853c04344aacc55afdd26f1fe" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 36277 } } peers { permanent_uuid: "0e4f629e966e419fb21b57161abdf46c" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 42531 } }
I20260619 10:17:37.678190 8819 leader_election.cc:288] T a2ab22162c954be09c0370e92d9369e5 P 0c4c7226ac8e48ca8d708ea8bf3b76f7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eed7c39853c04344aacc55afdd26f1fe (127.0.206.66:36277), 0e4f629e966e419fb21b57161abdf46c (127.0.206.67:42531)
I20260619 10:17:37.678447 8604 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a2ab22162c954be09c0370e92d9369e5" candidate_uuid: "0c4c7226ac8e48ca8d708ea8bf3b76f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eed7c39853c04344aacc55afdd26f1fe" is_pre_election: true
I20260619 10:17:37.678485 8744 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a2ab22162c954be09c0370e92d9369e5" candidate_uuid: "0c4c7226ac8e48ca8d708ea8bf3b76f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e4f629e966e419fb21b57161abdf46c" is_pre_election: true
I20260619 10:17:37.678557 8604 raft_consensus.cc:2454] T a2ab22162c954be09c0370e92d9369e5 P eed7c39853c04344aacc55afdd26f1fe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0c4c7226ac8e48ca8d708ea8bf3b76f7 in term 0.
I20260619 10:17:37.678591 8744 raft_consensus.cc:2454] T a2ab22162c954be09c0370e92d9369e5 P 0e4f629e966e419fb21b57161abdf46c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0c4c7226ac8e48ca8d708ea8bf3b76f7 in term 0.
I20260619 10:17:37.678766 8398 leader_election.cc:302] T a2ab22162c954be09c0370e92d9369e5 P 0c4c7226ac8e48ca8d708ea8bf3b76f7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0c4c7226ac8e48ca8d708ea8bf3b76f7, eed7c39853c04344aacc55afdd26f1fe; no voters:
I20260619 10:17:37.678875 8819 raft_consensus.cc:2781] T a2ab22162c954be09c0370e92d9369e5 P 0c4c7226ac8e48ca8d708ea8bf3b76f7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 10:17:37.678938 8819 raft_consensus.cc:482] T a2ab22162c954be09c0370e92d9369e5 P 0c4c7226ac8e48ca8d708ea8bf3b76f7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 10:17:37.678969 8819 raft_consensus.cc:3037] T a2ab22162c954be09c0370e92d9369e5 P 0c4c7226ac8e48ca8d708ea8bf3b76f7 [term 0 FOLLOWER]: Advancing to term 1
I20260619 10:17:37.679618 8819 raft_consensus.cc:504] T a2ab22162c954be09c0370e92d9369e5 P 0c4c7226ac8e48ca8d708ea8bf3b76f7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c4c7226ac8e48ca8d708ea8bf3b76f7" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 37567 } } peers { permanent_uuid: "eed7c39853c04344aacc55afdd26f1fe" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 36277 } } peers { permanent_uuid: "0e4f629e966e419fb21b57161abdf46c" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 42531 } }
I20260619 10:17:37.679744 8819 leader_election.cc:288] T a2ab22162c954be09c0370e92d9369e5 P 0c4c7226ac8e48ca8d708ea8bf3b76f7 [CANDIDATE]: Term 1 election: Requested vote from peers eed7c39853c04344aacc55afdd26f1fe (127.0.206.66:36277), 0e4f629e966e419fb21b57161abdf46c (127.0.206.67:42531)
I20260619 10:17:37.679905 8744 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a2ab22162c954be09c0370e92d9369e5" candidate_uuid: "0c4c7226ac8e48ca8d708ea8bf3b76f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e4f629e966e419fb21b57161abdf46c"
I20260619 10:17:37.679895 8604 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a2ab22162c954be09c0370e92d9369e5" candidate_uuid: "0c4c7226ac8e48ca8d708ea8bf3b76f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eed7c39853c04344aacc55afdd26f1fe"
I20260619 10:17:37.679977 8604 raft_consensus.cc:3037] T a2ab22162c954be09c0370e92d9369e5 P eed7c39853c04344aacc55afdd26f1fe [term 0 FOLLOWER]: Advancing to term 1
I20260619 10:17:37.679977 8744 raft_consensus.cc:3037] T a2ab22162c954be09c0370e92d9369e5 P 0e4f629e966e419fb21b57161abdf46c [term 0 FOLLOWER]: Advancing to term 1
I20260619 10:17:37.680720 8604 raft_consensus.cc:2454] T a2ab22162c954be09c0370e92d9369e5 P eed7c39853c04344aacc55afdd26f1fe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0c4c7226ac8e48ca8d708ea8bf3b76f7 in term 1.
I20260619 10:17:37.680720 8744 raft_consensus.cc:2454] T a2ab22162c954be09c0370e92d9369e5 P 0e4f629e966e419fb21b57161abdf46c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0c4c7226ac8e48ca8d708ea8bf3b76f7 in term 1.
I20260619 10:17:37.680953 8398 leader_election.cc:302] T a2ab22162c954be09c0370e92d9369e5 P 0c4c7226ac8e48ca8d708ea8bf3b76f7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0c4c7226ac8e48ca8d708ea8bf3b76f7, 0e4f629e966e419fb21b57161abdf46c; no voters:
I20260619 10:17:37.681030 8819 raft_consensus.cc:2781] T a2ab22162c954be09c0370e92d9369e5 P 0c4c7226ac8e48ca8d708ea8bf3b76f7 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 10:17:37.681107 8819 raft_consensus.cc:686] T a2ab22162c954be09c0370e92d9369e5 P 0c4c7226ac8e48ca8d708ea8bf3b76f7 [term 1 LEADER]: Becoming Leader. State: Replica: 0c4c7226ac8e48ca8d708ea8bf3b76f7, State: Running, Role: LEADER
I20260619 10:17:37.681164 8819 consensus_queue.cc:237] T a2ab22162c954be09c0370e92d9369e5 P 0c4c7226ac8e48ca8d708ea8bf3b76f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c4c7226ac8e48ca8d708ea8bf3b76f7" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 37567 } } peers { permanent_uuid: "eed7c39853c04344aacc55afdd26f1fe" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 36277 } } peers { permanent_uuid: "0e4f629e966e419fb21b57161abdf46c" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 42531 } }
I20260619 10:17:37.681448 8833 tablet_replica.cc:439] T a2ab22162c954be09c0370e92d9369e5 P 0c4c7226ac8e48ca8d708ea8bf3b76f7: TxnStatusTablet state changed. Reason: New leader 0c4c7226ac8e48ca8d708ea8bf3b76f7. Latest consensus state: current_term: 1 leader_uuid: "0c4c7226ac8e48ca8d708ea8bf3b76f7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c4c7226ac8e48ca8d708ea8bf3b76f7" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 37567 } } peers { permanent_uuid: "eed7c39853c04344aacc55afdd26f1fe" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 36277 } } peers { permanent_uuid: "0e4f629e966e419fb21b57161abdf46c" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 42531 } } }
I20260619 10:17:37.681543 8833 tablet_replica.cc:442] T a2ab22162c954be09c0370e92d9369e5 P 0c4c7226ac8e48ca8d708ea8bf3b76f7: This TxnStatusTablet replica's current role is: LEADER
I20260619 10:17:37.681746 8307 catalog_manager.cc:5495] T a2ab22162c954be09c0370e92d9369e5 P 0c4c7226ac8e48ca8d708ea8bf3b76f7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0c4c7226ac8e48ca8d708ea8bf3b76f7 (127.0.206.65). New cstate: current_term: 1 leader_uuid: "0c4c7226ac8e48ca8d708ea8bf3b76f7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c4c7226ac8e48ca8d708ea8bf3b76f7" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 37567 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "eed7c39853c04344aacc55afdd26f1fe" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 36277 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0e4f629e966e419fb21b57161abdf46c" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 42531 } health_report { overall_health: UNKNOWN } } }
I20260619 10:17:37.682049 8857 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260619 10:17:37.682179 8857 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260619 10:17:37.699990 8813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:37.696575 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43068 (local address 127.0.206.65:37567)
0619 10:17:37.696617 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0619 10:17:37.696619 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 10:17:37.696827 (+ 208us) server_negotiation.cc:407] Connection header received
0619 10:17:37.696905 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:17:37.696906 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:17:37.696917 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:17:37.696975 (+ 58us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 10:17:37.697152 (+ 177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:37.698122 (+ 970us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:37.699132 (+ 1010us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:37.699425 (+ 293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:37.699498 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:37.699707 (+ 209us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 10:17:37.699739 (+ 32us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 10:17:37.699778 (+ 39us) server_negotiation.cc:1035] Waiting for connection context
0619 10:17:37.699882 (+ 104us) server_negotiation.cc:299] Negotiation successful
0619 10:17:37.699909 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260619 10:17:37.699994 8852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:37.696664 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.65:37567 (local address 127.0.0.1:43068)
0619 10:17:37.696741 (+ 77us) negotiation.cc:107] Waiting for socket to connect
0619 10:17:37.696756 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0619 10:17:37.696828 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 10:17:37.696980 (+ 152us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 10:17:37.696981 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 10:17:37.696983 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0619 10:17:37.697047 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:37.697050 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:37.698223 (+ 1173us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:37.698224 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:37.699060 (+ 836us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:37.699062 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:37.699533 (+ 471us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:37.699534 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:37.699599 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:37.699614 (+ 15us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0619 10:17:37.699798 (+ 184us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0619 10:17:37.699804 (+ 6us) client_negotiation.cc:769] Sending connection context
0619 10:17:37.699868 (+ 64us) client_negotiation.cc:240] Negotiation successful
0619 10:17:37.699911 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35}
I20260619 10:17:37.701537 8744 raft_consensus.cc:1261] T a2ab22162c954be09c0370e92d9369e5 P 0e4f629e966e419fb21b57161abdf46c [term 1 FOLLOWER]: Refusing update from remote peer 0c4c7226ac8e48ca8d708ea8bf3b76f7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 10:17:37.701761 8819 consensus_queue.cc:1035] T a2ab22162c954be09c0370e92d9369e5 P 0c4c7226ac8e48ca8d708ea8bf3b76f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e4f629e966e419fb21b57161abdf46c" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 42531 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 10:17:37.702690 8825 tablet_replica.cc:439] T a2ab22162c954be09c0370e92d9369e5 P 0c4c7226ac8e48ca8d708ea8bf3b76f7: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "0c4c7226ac8e48ca8d708ea8bf3b76f7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c4c7226ac8e48ca8d708ea8bf3b76f7" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 37567 } } peers { permanent_uuid: "eed7c39853c04344aacc55afdd26f1fe" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 36277 } } peers { permanent_uuid: "0e4f629e966e419fb21b57161abdf46c" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 42531 } } }
I20260619 10:17:37.702772 8825 tablet_replica.cc:442] T a2ab22162c954be09c0370e92d9369e5 P 0c4c7226ac8e48ca8d708ea8bf3b76f7: This TxnStatusTablet replica's current role is: LEADER
I20260619 10:17:37.703267 8861 tablet_replica.cc:439] T a2ab22162c954be09c0370e92d9369e5 P 0e4f629e966e419fb21b57161abdf46c: TxnStatusTablet state changed. Reason: New leader 0c4c7226ac8e48ca8d708ea8bf3b76f7. Latest consensus state: current_term: 1 leader_uuid: "0c4c7226ac8e48ca8d708ea8bf3b76f7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c4c7226ac8e48ca8d708ea8bf3b76f7" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 37567 } } peers { permanent_uuid: "eed7c39853c04344aacc55afdd26f1fe" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 36277 } } peers { permanent_uuid: "0e4f629e966e419fb21b57161abdf46c" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 42531 } } }
I20260619 10:17:37.703414 8861 tablet_replica.cc:442] T a2ab22162c954be09c0370e92d9369e5 P 0e4f629e966e419fb21b57161abdf46c: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 10:17:37.703495 8861 tablet_replica.cc:439] T a2ab22162c954be09c0370e92d9369e5 P 0e4f629e966e419fb21b57161abdf46c: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "0c4c7226ac8e48ca8d708ea8bf3b76f7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c4c7226ac8e48ca8d708ea8bf3b76f7" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 37567 } } peers { permanent_uuid: "eed7c39853c04344aacc55afdd26f1fe" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 36277 } } peers { permanent_uuid: "0e4f629e966e419fb21b57161abdf46c" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 42531 } } }
I20260619 10:17:37.703557 8861 tablet_replica.cc:442] T a2ab22162c954be09c0370e92d9369e5 P 0e4f629e966e419fb21b57161abdf46c: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 10:17:37.705021 8604 raft_consensus.cc:1261] T a2ab22162c954be09c0370e92d9369e5 P eed7c39853c04344aacc55afdd26f1fe [term 1 FOLLOWER]: Refusing update from remote peer 0c4c7226ac8e48ca8d708ea8bf3b76f7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 10:17:37.705915 8833 consensus_queue.cc:1035] T a2ab22162c954be09c0370e92d9369e5 P 0c4c7226ac8e48ca8d708ea8bf3b76f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eed7c39853c04344aacc55afdd26f1fe" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 36277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 10:17:37.706401 8845 tablet_replica.cc:439] T a2ab22162c954be09c0370e92d9369e5 P eed7c39853c04344aacc55afdd26f1fe: TxnStatusTablet state changed. Reason: New leader 0c4c7226ac8e48ca8d708ea8bf3b76f7. Latest consensus state: current_term: 1 leader_uuid: "0c4c7226ac8e48ca8d708ea8bf3b76f7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c4c7226ac8e48ca8d708ea8bf3b76f7" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 37567 } } peers { permanent_uuid: "eed7c39853c04344aacc55afdd26f1fe" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 36277 } } peers { permanent_uuid: "0e4f629e966e419fb21b57161abdf46c" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 42531 } } }
I20260619 10:17:37.706503 8845 tablet_replica.cc:442] T a2ab22162c954be09c0370e92d9369e5 P eed7c39853c04344aacc55afdd26f1fe: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 10:17:37.706537 8845 tablet_replica.cc:439] T a2ab22162c954be09c0370e92d9369e5 P eed7c39853c04344aacc55afdd26f1fe: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "0c4c7226ac8e48ca8d708ea8bf3b76f7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c4c7226ac8e48ca8d708ea8bf3b76f7" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 37567 } } peers { permanent_uuid: "eed7c39853c04344aacc55afdd26f1fe" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 36277 } } peers { permanent_uuid: "0e4f629e966e419fb21b57161abdf46c" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 42531 } } }
I20260619 10:17:37.706576 8845 tablet_replica.cc:442] T a2ab22162c954be09c0370e92d9369e5 P eed7c39853c04344aacc55afdd26f1fe: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 10:17:37.708446 8812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:37.705417 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36310 (local address 127.0.206.66:36277)
0619 10:17:37.705457 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0619 10:17:37.705459 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 10:17:37.705466 (+ 7us) server_negotiation.cc:407] Connection header received
0619 10:17:37.705499 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:17:37.705499 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:17:37.705519 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:17:37.705581 (+ 62us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 10:17:37.705735 (+ 154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:37.706456 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:37.707557 (+ 1101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:37.707892 (+ 335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:37.707965 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:37.708097 (+ 132us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 10:17:37.708233 (+ 136us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 10:17:37.708295 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0619 10:17:37.708350 (+ 55us) server_negotiation.cc:299] Negotiation successful
0619 10:17:37.708368 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260619 10:17:37.713949 8813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:37.710760 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.66:54781 (local address 127.0.206.65:37567)
0619 10:17:37.710801 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0619 10:17:37.710803 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 10:17:37.710961 (+ 158us) server_negotiation.cc:407] Connection header received
0619 10:17:37.711038 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:17:37.711039 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:17:37.711050 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:17:37.711104 (+ 54us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 10:17:37.711239 (+ 135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:37.711965 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:37.713185 (+ 1220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:37.713485 (+ 300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:37.713547 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:37.713719 (+ 172us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 10:17:37.713750 (+ 31us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 10:17:37.713793 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0619 10:17:37.713856 (+ 63us) server_negotiation.cc:299] Negotiation successful
0619 10:17:37.713883 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260619 10:17:37.714149 8865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:37.710709 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.65:37567 (local address 127.0.206.66:54781)
0619 10:17:37.710889 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0619 10:17:37.710899 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 10:17:37.710974 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 10:17:37.711107 (+ 133us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 10:17:37.711107 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 10:17:37.711109 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0619 10:17:37.711168 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:37.711170 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:37.712040 (+ 870us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:37.712041 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:37.713113 (+ 1072us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:37.713115 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:37.713581 (+ 466us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:37.713582 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:37.713627 (+ 45us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:37.713639 (+ 12us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0619 10:17:37.713804 (+ 165us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0619 10:17:37.713805 (+ 1us) client_negotiation.cc:769] Sending connection context
0619 10:17:37.713852 (+ 47us) client_negotiation.cc:240] Negotiation successful
0619 10:17:37.713898 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":94,"threads_started":1}
I20260619 10:17:37.714766 8819 tablet_replica.cc:439] T a2ab22162c954be09c0370e92d9369e5 P 0c4c7226ac8e48ca8d708ea8bf3b76f7: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "0c4c7226ac8e48ca8d708ea8bf3b76f7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c4c7226ac8e48ca8d708ea8bf3b76f7" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 37567 } } peers { permanent_uuid: "eed7c39853c04344aacc55afdd26f1fe" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 36277 } } peers { permanent_uuid: "0e4f629e966e419fb21b57161abdf46c" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 42531 } } }
I20260619 10:17:37.714849 8819 tablet_replica.cc:442] T a2ab22162c954be09c0370e92d9369e5 P 0c4c7226ac8e48ca8d708ea8bf3b76f7: This TxnStatusTablet replica's current role is: LEADER
I20260619 10:17:37.716226 8744 raft_consensus.cc:1261] T dc0c7f2b3661431896754d048cba6f5a P 0e4f629e966e419fb21b57161abdf46c [term 1 FOLLOWER]: Refusing update from remote peer eed7c39853c04344aacc55afdd26f1fe: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 10:17:37.716329 8464 raft_consensus.cc:1261] T dc0c7f2b3661431896754d048cba6f5a P 0c4c7226ac8e48ca8d708ea8bf3b76f7 [term 1 FOLLOWER]: Refusing update from remote peer eed7c39853c04344aacc55afdd26f1fe: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 10:17:37.716471 8845 consensus_queue.cc:1035] T dc0c7f2b3661431896754d048cba6f5a P eed7c39853c04344aacc55afdd26f1fe [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e4f629e966e419fb21b57161abdf46c" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 42531 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 10:17:37.716638 8845 consensus_queue.cc:1035] T dc0c7f2b3661431896754d048cba6f5a P eed7c39853c04344aacc55afdd26f1fe [LEADER]: Connected to new peer: Peer: permanent_uuid: "0c4c7226ac8e48ca8d708ea8bf3b76f7" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 37567 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 10:17:37.725031 8812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:37.722471 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.65:60319 (local address 127.0.206.66:36277)
0619 10:17:37.722562 (+ 91us) server_negotiation.cc:206] Beginning negotiation
0619 10:17:37.722564 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 10:17:37.722570 (+ 6us) server_negotiation.cc:407] Connection header received
0619 10:17:37.722588 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:17:37.722589 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:17:37.722624 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:17:37.722672 (+ 48us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 10:17:37.722770 (+ 98us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:37.723213 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:37.724083 (+ 870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:37.724394 (+ 311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:37.724431 (+ 37us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:37.724778 (+ 347us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 10:17:37.724811 (+ 33us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 10:17:37.724871 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0619 10:17:37.724949 (+ 78us) server_negotiation.cc:299] Negotiation successful
0619 10:17:37.724966 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260619 10:17:37.725057 8870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:37.722264 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.66:36277 (local address 127.0.206.65:60319)
0619 10:17:37.722403 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0619 10:17:37.722414 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 10:17:37.722459 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 10:17:37.722675 (+ 216us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 10:17:37.722676 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 10:17:37.722678 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0619 10:17:37.722720 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:37.722722 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:37.723284 (+ 562us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:37.723285 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:37.723978 (+ 693us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:37.723981 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:37.724505 (+ 524us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:37.724506 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:37.724651 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:37.724667 (+ 16us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0619 10:17:37.724894 (+ 227us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0619 10:17:37.724896 (+ 2us) client_negotiation.cc:769] Sending connection context
0619 10:17:37.724938 (+ 42us) client_negotiation.cc:240] Negotiation successful
0619 10:17:37.724980 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":78,"threads_started":1}
I20260619 10:17:37.739506 8306 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:58402:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 10:17:37.739607 8306 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:58402:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 10:17:37.740185 8306 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f99ceb70a6684009b753712e007c5de5: Sending DeleteTablet for 3 replicas of tablet dc0c7f2b3661431896754d048cba6f5a
I20260619 10:17:37.740406 8584 tablet_service.cc:1514] Processing DeleteTablet for tablet dc0c7f2b3661431896754d048cba6f5a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 10:17:37 UTC) from {username='oryx', principal='oryx/127.0.206.126@KRBTEST.COM'} at 127.0.0.1:36298
I20260619 10:17:37.740439 8724 tablet_service.cc:1514] Processing DeleteTablet for tablet dc0c7f2b3661431896754d048cba6f5a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 10:17:37 UTC) from {username='oryx', principal='oryx/127.0.206.126@KRBTEST.COM'} at 127.0.0.1:36834
I20260619 10:17:37.740545 8839 tablet_replica.cc:330] T dc0c7f2b3661431896754d048cba6f5a P eed7c39853c04344aacc55afdd26f1fe: stopping tablet replica
I20260619 10:17:37.740602 8839 raft_consensus.cc:2229] T dc0c7f2b3661431896754d048cba6f5a P eed7c39853c04344aacc55afdd26f1fe [term 1 LEADER]: Raft consensus shutting down.
I20260619 10:17:37.740696 8839 raft_consensus.cc:2258] T dc0c7f2b3661431896754d048cba6f5a P eed7c39853c04344aacc55afdd26f1fe [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 10:17:37.740763 8840 tablet_replica.cc:330] T dc0c7f2b3661431896754d048cba6f5a P 0e4f629e966e419fb21b57161abdf46c: stopping tablet replica
I20260619 10:17:37.740813 8840 raft_consensus.cc:2229] T dc0c7f2b3661431896754d048cba6f5a P 0e4f629e966e419fb21b57161abdf46c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 10:17:37.740849 8840 raft_consensus.cc:2258] T dc0c7f2b3661431896754d048cba6f5a P 0e4f629e966e419fb21b57161abdf46c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 10:17:37.741181 8839 ts_tablet_manager.cc:1905] T dc0c7f2b3661431896754d048cba6f5a P eed7c39853c04344aacc55afdd26f1fe: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 10:17:37.741186 8840 ts_tablet_manager.cc:1905] T dc0c7f2b3661431896754d048cba6f5a P 0e4f629e966e419fb21b57161abdf46c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 10:17:37.742791 8444 tablet_service.cc:1514] Processing DeleteTablet for tablet dc0c7f2b3661431896754d048cba6f5a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 10:17:37 UTC) from {username='oryx', principal='oryx/127.0.206.126@KRBTEST.COM'} at 127.0.0.1:43058
I20260619 10:17:37.742877 8838 tablet_replica.cc:330] T dc0c7f2b3661431896754d048cba6f5a P 0c4c7226ac8e48ca8d708ea8bf3b76f7: stopping tablet replica
I20260619 10:17:37.742919 8838 raft_consensus.cc:2229] T dc0c7f2b3661431896754d048cba6f5a P 0c4c7226ac8e48ca8d708ea8bf3b76f7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 10:17:37.742950 8838 raft_consensus.cc:2258] T dc0c7f2b3661431896754d048cba6f5a P 0c4c7226ac8e48ca8d708ea8bf3b76f7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 10:17:37.742950 8840 ts_tablet_manager.cc:1918] T dc0c7f2b3661431896754d048cba6f5a P 0e4f629e966e419fb21b57161abdf46c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260619 10:17:37.742998 8840 log.cc:1196] T dc0c7f2b3661431896754d048cba6f5a P 0e4f629e966e419fb21b57161abdf46c: Deleting WAL directory at /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781864224577058-825-0/minicluster-data/ts-2/wal/wals/dc0c7f2b3661431896754d048cba6f5a
I20260619 10:17:37.743275 8840 ts_tablet_manager.cc:1939] T dc0c7f2b3661431896754d048cba6f5a P 0e4f629e966e419fb21b57161abdf46c: Deleting consensus metadata
I20260619 10:17:37.743585 8294 catalog_manager.cc:4841] TS 0e4f629e966e419fb21b57161abdf46c (127.0.206.67:42531): tablet dc0c7f2b3661431896754d048cba6f5a (table test-table [id=7940c309a7d64070b673aa22a36cfb11]) successfully deleted
I20260619 10:17:37.745301 8838 ts_tablet_manager.cc:1905] T dc0c7f2b3661431896754d048cba6f5a P 0c4c7226ac8e48ca8d708ea8bf3b76f7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 10:17:37.746779 8838 ts_tablet_manager.cc:1918] T dc0c7f2b3661431896754d048cba6f5a P 0c4c7226ac8e48ca8d708ea8bf3b76f7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260619 10:17:37.746837 8838 log.cc:1196] T dc0c7f2b3661431896754d048cba6f5a P 0c4c7226ac8e48ca8d708ea8bf3b76f7: Deleting WAL directory at /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781864224577058-825-0/minicluster-data/ts-0/wal/wals/dc0c7f2b3661431896754d048cba6f5a
I20260619 10:17:37.747040 8838 ts_tablet_manager.cc:1939] T dc0c7f2b3661431896754d048cba6f5a P 0c4c7226ac8e48ca8d708ea8bf3b76f7: Deleting consensus metadata
I20260619 10:17:37.747709 8839 ts_tablet_manager.cc:1918] T dc0c7f2b3661431896754d048cba6f5a P eed7c39853c04344aacc55afdd26f1fe: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260619 10:17:37.747772 8839 log.cc:1196] T dc0c7f2b3661431896754d048cba6f5a P eed7c39853c04344aacc55afdd26f1fe: Deleting WAL directory at /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781864224577058-825-0/minicluster-data/ts-1/wal/wals/dc0c7f2b3661431896754d048cba6f5a
I20260619 10:17:37.747983 8839 ts_tablet_manager.cc:1939] T dc0c7f2b3661431896754d048cba6f5a P eed7c39853c04344aacc55afdd26f1fe: Deleting consensus metadata
I20260619 10:17:37.748265 8294 catalog_manager.cc:4841] TS eed7c39853c04344aacc55afdd26f1fe (127.0.206.66:36277): tablet dc0c7f2b3661431896754d048cba6f5a (table test-table [id=7940c309a7d64070b673aa22a36cfb11]) successfully deleted
I20260619 10:17:37.748524 8293 catalog_manager.cc:4841] TS 0c4c7226ac8e48ca8d708ea8bf3b76f7 (127.0.206.65:37567): tablet dc0c7f2b3661431896754d048cba6f5a (table test-table [id=7940c309a7d64070b673aa22a36cfb11]) successfully deleted
Jun 19 10:17:37 dist-test-slave-6pt2 krb5kdc[8262](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864257, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Jun 19 10:17:37 dist-test-slave-6pt2 krb5kdc[8262](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.0.206.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 10:17:37 dist-test-slave-6pt2 krb5kdc[8262](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.0.206.126@KRBTEST.COM, Server not found in Kerberos database
W20260619 10:17:37.771729 8356 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 10:17:37.765116 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58428 (local address 127.0.206.126:45657)
0619 10:17:37.765158 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0619 10:17:37.765160 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 10:17:37.765293 (+ 133us) server_negotiation.cc:407] Connection header received
0619 10:17:37.765359 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:17:37.765360 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:17:37.765370 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:17:37.765432 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:17:37.766118 (+ 686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:37.766907 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:37.768032 (+ 1125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:37.768316 (+ 284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:37.768388 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:37.771657 (+ 3269us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58428: BlockingRecv error: recv got EOF from 127.0.0.1:58428 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
Jun 19 10:17:37 dist-test-slave-6pt2 krb5kdc[8262](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864257, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.0.206.126@KRBTEST.COM
I20260619 10:17:37.781162 8356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:37.772774 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58442 (local address 127.0.206.126:45657)
0619 10:17:37.772826 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0619 10:17:37.772827 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 10:17:37.772836 (+ 9us) server_negotiation.cc:407] Connection header received
0619 10:17:37.772897 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:17:37.772898 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:17:37.772907 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:17:37.772963 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:17:37.773555 (+ 592us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:37.774304 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:37.775429 (+ 1125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:37.775723 (+ 294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:37.775783 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:37.778299 (+ 2516us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 10:17:37.778300 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 10:17:37.778300 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 10:17:37.778307 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 10:17:37.779965 (+ 1658us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:17:37.780340 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:17:37.780342 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:17:37.780342 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:17:37.780400 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:17:37.780652 (+ 252us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:17:37.780652 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:17:37.780653 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:17:37.780780 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 10:17:37.780926 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0619 10:17:37.781003 (+ 77us) server_negotiation.cc:299] Negotiation successful
0619 10:17:37.781074 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260619 10:17:37.783005 8306 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58442:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 10:17:37.783179 8306 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 10:17:37.784534 8444 tablet_service.cc:1467] Processing CreateTablet for tablet 3911676e1c43437489d614d1c544ec86 (DEFAULT_TABLE table=test-table [id=96e3553b61a24cfeb0addbe4e573902d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 10:17:37.784673 8444 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3911676e1c43437489d614d1c544ec86. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 10:17:37.785985 8584 tablet_service.cc:1467] Processing CreateTablet for tablet 3911676e1c43437489d614d1c544ec86 (DEFAULT_TABLE table=test-table [id=96e3553b61a24cfeb0addbe4e573902d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 10:17:37.786092 8584 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3911676e1c43437489d614d1c544ec86. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 10:17:37.787333 8815 tablet_bootstrap.cc:492] T 3911676e1c43437489d614d1c544ec86 P 0c4c7226ac8e48ca8d708ea8bf3b76f7: Bootstrap starting.
I20260619 10:17:37.787634 8815 tablet_bootstrap.cc:654] T 3911676e1c43437489d614d1c544ec86 P 0c4c7226ac8e48ca8d708ea8bf3b76f7: Neither blocks nor log segments found. Creating new log.
I20260619 10:17:37.787837 8820 tablet_bootstrap.cc:492] T 3911676e1c43437489d614d1c544ec86 P eed7c39853c04344aacc55afdd26f1fe: Bootstrap starting.
I20260619 10:17:37.788223 8820 tablet_bootstrap.cc:654] T 3911676e1c43437489d614d1c544ec86 P eed7c39853c04344aacc55afdd26f1fe: Neither blocks nor log segments found. Creating new log.
I20260619 10:17:37.788614 8724 tablet_service.cc:1467] Processing CreateTablet for tablet 3911676e1c43437489d614d1c544ec86 (DEFAULT_TABLE table=test-table [id=96e3553b61a24cfeb0addbe4e573902d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 10:17:37.788731 8724 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3911676e1c43437489d614d1c544ec86. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 10:17:37.790058 8816 tablet_bootstrap.cc:492] T 3911676e1c43437489d614d1c544ec86 P 0e4f629e966e419fb21b57161abdf46c: Bootstrap starting.
I20260619 10:17:37.790411 8816 tablet_bootstrap.cc:654] T 3911676e1c43437489d614d1c544ec86 P 0e4f629e966e419fb21b57161abdf46c: Neither blocks nor log segments found. Creating new log.
I20260619 10:17:37.790988 8816 tablet_bootstrap.cc:492] T 3911676e1c43437489d614d1c544ec86 P 0e4f629e966e419fb21b57161abdf46c: No bootstrap required, opened a new log
I20260619 10:17:37.791046 8816 ts_tablet_manager.cc:1397] T 3911676e1c43437489d614d1c544ec86 P 0e4f629e966e419fb21b57161abdf46c: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260619 10:17:37.791183 8816 raft_consensus.cc:348] T 3911676e1c43437489d614d1c544ec86 P 0e4f629e966e419fb21b57161abdf46c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c4c7226ac8e48ca8d708ea8bf3b76f7" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 37567 } } peers { permanent_uuid: "eed7c39853c04344aacc55afdd26f1fe" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 36277 } } peers { permanent_uuid: "0e4f629e966e419fb21b57161abdf46c" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 42531 } }
I20260619 10:17:37.791258 8816 raft_consensus.cc:374] T 3911676e1c43437489d614d1c544ec86 P 0e4f629e966e419fb21b57161abdf46c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 10:17:37.791277 8816 raft_consensus.cc:729] T 3911676e1c43437489d614d1c544ec86 P 0e4f629e966e419fb21b57161abdf46c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e4f629e966e419fb21b57161abdf46c, State: Initialized, Role: FOLLOWER
I20260619 10:17:37.791275 8815 tablet_bootstrap.cc:492] T 3911676e1c43437489d614d1c544ec86 P 0c4c7226ac8e48ca8d708ea8bf3b76f7: No bootstrap required, opened a new log
I20260619 10:17:37.791318 8815 ts_tablet_manager.cc:1397] T 3911676e1c43437489d614d1c544ec86 P 0c4c7226ac8e48ca8d708ea8bf3b76f7: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20260619 10:17:37.791314 8816 consensus_queue.cc:260] T 3911676e1c43437489d614d1c544ec86 P 0e4f629e966e419fb21b57161abdf46c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c4c7226ac8e48ca8d708ea8bf3b76f7" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 37567 } } peers { permanent_uuid: "eed7c39853c04344aacc55afdd26f1fe" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 36277 } } peers { permanent_uuid: "0e4f629e966e419fb21b57161abdf46c" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 42531 } }
I20260619 10:17:37.791390 8816 ts_tablet_manager.cc:1428] T 3911676e1c43437489d614d1c544ec86 P 0e4f629e966e419fb21b57161abdf46c: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260619 10:17:37.791414 8815 raft_consensus.cc:348] T 3911676e1c43437489d614d1c544ec86 P 0c4c7226ac8e48ca8d708ea8bf3b76f7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c4c7226ac8e48ca8d708ea8bf3b76f7" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 37567 } } peers { permanent_uuid: "eed7c39853c04344aacc55afdd26f1fe" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 36277 } } peers { permanent_uuid: "0e4f629e966e419fb21b57161abdf46c" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 42531 } }
I20260619 10:17:37.791486 8815 raft_consensus.cc:374] T 3911676e1c43437489d614d1c544ec86 P 0c4c7226ac8e48ca8d708ea8bf3b76f7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 10:17:37.791507 8815 raft_consensus.cc:729] T 3911676e1c43437489d614d1c544ec86 P 0c4c7226ac8e48ca8d708ea8bf3b76f7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0c4c7226ac8e48ca8d708ea8bf3b76f7, State: Initialized, Role: FOLLOWER
I20260619 10:17:37.791548 8815 consensus_queue.cc:260] T 3911676e1c43437489d614d1c544ec86 P 0c4c7226ac8e48ca8d708ea8bf3b76f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c4c7226ac8e48ca8d708ea8bf3b76f7" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 37567 } } peers { permanent_uuid: "eed7c39853c04344aacc55afdd26f1fe" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 36277 } } peers { permanent_uuid: "0e4f629e966e419fb21b57161abdf46c" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 42531 } }
I20260619 10:17:37.791630 8815 ts_tablet_manager.cc:1428] T 3911676e1c43437489d614d1c544ec86 P 0c4c7226ac8e48ca8d708ea8bf3b76f7: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260619 10:17:37.792338 8820 tablet_bootstrap.cc:492] T 3911676e1c43437489d614d1c544ec86 P eed7c39853c04344aacc55afdd26f1fe: No bootstrap required, opened a new log
I20260619 10:17:37.792394 8820 ts_tablet_manager.cc:1397] T 3911676e1c43437489d614d1c544ec86 P eed7c39853c04344aacc55afdd26f1fe: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.000s
I20260619 10:17:37.792554 8820 raft_consensus.cc:348] T 3911676e1c43437489d614d1c544ec86 P eed7c39853c04344aacc55afdd26f1fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c4c7226ac8e48ca8d708ea8bf3b76f7" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 37567 } } peers { permanent_uuid: "eed7c39853c04344aacc55afdd26f1fe" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 36277 } } peers { permanent_uuid: "0e4f629e966e419fb21b57161abdf46c" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 42531 } }
I20260619 10:17:37.792629 8820 raft_consensus.cc:374] T 3911676e1c43437489d614d1c544ec86 P eed7c39853c04344aacc55afdd26f1fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 10:17:37.792649 8820 raft_consensus.cc:729] T 3911676e1c43437489d614d1c544ec86 P eed7c39853c04344aacc55afdd26f1fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eed7c39853c04344aacc55afdd26f1fe, State: Initialized, Role: FOLLOWER
I20260619 10:17:37.792691 8820 consensus_queue.cc:260] T 3911676e1c43437489d614d1c544ec86 P eed7c39853c04344aacc55afdd26f1fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c4c7226ac8e48ca8d708ea8bf3b76f7" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 37567 } } peers { permanent_uuid: "eed7c39853c04344aacc55afdd26f1fe" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 36277 } } peers { permanent_uuid: "0e4f629e966e419fb21b57161abdf46c" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 42531 } }
I20260619 10:17:37.792774 8820 ts_tablet_manager.cc:1428] T 3911676e1c43437489d614d1c544ec86 P eed7c39853c04344aacc55afdd26f1fe: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260619 10:17:37.985749 8821 raft_consensus.cc:482] T 3911676e1c43437489d614d1c544ec86 P 0e4f629e966e419fb21b57161abdf46c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 10:17:37.985880 8821 raft_consensus.cc:504] T 3911676e1c43437489d614d1c544ec86 P 0e4f629e966e419fb21b57161abdf46c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c4c7226ac8e48ca8d708ea8bf3b76f7" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 37567 } } peers { permanent_uuid: "eed7c39853c04344aacc55afdd26f1fe" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 36277 } } peers { permanent_uuid: "0e4f629e966e419fb21b57161abdf46c" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 42531 } }
I20260619 10:17:37.986255 8821 leader_election.cc:288] T 3911676e1c43437489d614d1c544ec86 P 0e4f629e966e419fb21b57161abdf46c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0c4c7226ac8e48ca8d708ea8bf3b76f7 (127.0.206.65:37567), eed7c39853c04344aacc55afdd26f1fe (127.0.206.66:36277)
I20260619 10:17:37.989974 8889 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:37.986446 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.65:37567 (local address 127.0.206.67:49547)
0619 10:17:37.986569 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0619 10:17:37.986580 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 10:17:37.986665 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 10:17:37.986845 (+ 180us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 10:17:37.986845 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 10:17:37.986848 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 10:17:37.986948 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:37.986951 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:37.987873 (+ 922us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:37.987874 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:37.989197 (+ 1323us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:37.989200 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:37.989783 (+ 583us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:37.989783 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:37.989825 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:37.989828 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 10:17:37.989870 (+ 42us) client_negotiation.cc:240] Negotiation successful
0619 10:17:37.989901 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":105,"mutex_wait_us":53,"thread_start_us":70,"threads_started":1}
I20260619 10:17:37.989986 8813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:37.986578 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.67:49547 (local address 127.0.206.65:37567)
0619 10:17:37.986640 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0619 10:17:37.986642 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 10:17:37.986651 (+ 9us) server_negotiation.cc:407] Connection header received
0619 10:17:37.986762 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:17:37.986762 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:17:37.986771 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:17:37.986826 (+ 55us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 10:17:37.987044 (+ 218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:37.987774 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:37.989283 (+ 1509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:37.989705 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:37.989774 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:37.989782 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0619 10:17:37.989886 (+ 104us) server_negotiation.cc:299] Negotiation successful
0619 10:17:37.989919 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260619 10:17:37.990214 8464 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3911676e1c43437489d614d1c544ec86" candidate_uuid: "0e4f629e966e419fb21b57161abdf46c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0c4c7226ac8e48ca8d708ea8bf3b76f7" is_pre_election: true
I20260619 10:17:37.990325 8464 raft_consensus.cc:2454] T 3911676e1c43437489d614d1c544ec86 P 0c4c7226ac8e48ca8d708ea8bf3b76f7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e4f629e966e419fb21b57161abdf46c in term 0.
I20260619 10:17:37.990476 8890 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:37.986447 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.66:36277 (local address 127.0.206.67:34799)
0619 10:17:37.986585 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0619 10:17:37.986594 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0619 10:17:37.986678 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 10:17:37.986844 (+ 166us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 10:17:37.986844 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 10:17:37.986847 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 10:17:37.986963 (+ 116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:37.986965 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:37.987855 (+ 890us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:37.987856 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:37.989459 (+ 1603us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:37.989461 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:37.990176 (+ 715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:37.990177 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:37.990262 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:37.990267 (+ 5us) client_negotiation.cc:769] Sending connection context
0619 10:17:37.990323 (+ 56us) client_negotiation.cc:240] Negotiation successful
0619 10:17:37.990377 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"mutex_wait_us":76,"thread_start_us":56,"threads_started":1}
I20260619 10:17:37.990473 8812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:37.986515 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.67:34799 (local address 127.0.206.66:36277)
0619 10:17:37.986569 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0619 10:17:37.986571 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 10:17:37.986686 (+ 115us) server_negotiation.cc:407] Connection header received
0619 10:17:37.986738 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:17:37.986739 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:17:37.986749 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:17:37.986826 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 10:17:37.987034 (+ 208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:37.987774 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:37.989537 (+ 1763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:37.990033 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:37.990101 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:37.990110 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0619 10:17:37.990362 (+ 252us) server_negotiation.cc:299] Negotiation successful
0619 10:17:37.990403 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260619 10:17:37.990494 8677 leader_election.cc:302] T 3911676e1c43437489d614d1c544ec86 P 0e4f629e966e419fb21b57161abdf46c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0c4c7226ac8e48ca8d708ea8bf3b76f7, 0e4f629e966e419fb21b57161abdf46c; no voters:
I20260619 10:17:37.990605 8821 raft_consensus.cc:2781] T 3911676e1c43437489d614d1c544ec86 P 0e4f629e966e419fb21b57161abdf46c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 10:17:37.990654 8821 raft_consensus.cc:482] T 3911676e1c43437489d614d1c544ec86 P 0e4f629e966e419fb21b57161abdf46c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 10:17:37.990684 8821 raft_consensus.cc:3037] T 3911676e1c43437489d614d1c544ec86 P 0e4f629e966e419fb21b57161abdf46c [term 0 FOLLOWER]: Advancing to term 1
I20260619 10:17:37.990779 8604 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3911676e1c43437489d614d1c544ec86" candidate_uuid: "0e4f629e966e419fb21b57161abdf46c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eed7c39853c04344aacc55afdd26f1fe" is_pre_election: true
I20260619 10:17:37.990855 8604 raft_consensus.cc:2454] T 3911676e1c43437489d614d1c544ec86 P eed7c39853c04344aacc55afdd26f1fe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e4f629e966e419fb21b57161abdf46c in term 0.
I20260619 10:17:37.991472 8821 raft_consensus.cc:504] T 3911676e1c43437489d614d1c544ec86 P 0e4f629e966e419fb21b57161abdf46c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c4c7226ac8e48ca8d708ea8bf3b76f7" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 37567 } } peers { permanent_uuid: "eed7c39853c04344aacc55afdd26f1fe" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 36277 } } peers { permanent_uuid: "0e4f629e966e419fb21b57161abdf46c" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 42531 } }
I20260619 10:17:37.991611 8821 leader_election.cc:288] T 3911676e1c43437489d614d1c544ec86 P 0e4f629e966e419fb21b57161abdf46c [CANDIDATE]: Term 1 election: Requested vote from peers 0c4c7226ac8e48ca8d708ea8bf3b76f7 (127.0.206.65:37567), eed7c39853c04344aacc55afdd26f1fe (127.0.206.66:36277)
I20260619 10:17:37.991782 8604 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3911676e1c43437489d614d1c544ec86" candidate_uuid: "0e4f629e966e419fb21b57161abdf46c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eed7c39853c04344aacc55afdd26f1fe"
I20260619 10:17:37.991799 8464 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3911676e1c43437489d614d1c544ec86" candidate_uuid: "0e4f629e966e419fb21b57161abdf46c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0c4c7226ac8e48ca8d708ea8bf3b76f7"
I20260619 10:17:37.991855 8604 raft_consensus.cc:3037] T 3911676e1c43437489d614d1c544ec86 P eed7c39853c04344aacc55afdd26f1fe [term 0 FOLLOWER]: Advancing to term 1
I20260619 10:17:37.991863 8464 raft_consensus.cc:3037] T 3911676e1c43437489d614d1c544ec86 P 0c4c7226ac8e48ca8d708ea8bf3b76f7 [term 0 FOLLOWER]: Advancing to term 1
I20260619 10:17:37.992604 8464 raft_consensus.cc:2454] T 3911676e1c43437489d614d1c544ec86 P 0c4c7226ac8e48ca8d708ea8bf3b76f7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e4f629e966e419fb21b57161abdf46c in term 1.
I20260619 10:17:37.992604 8604 raft_consensus.cc:2454] T 3911676e1c43437489d614d1c544ec86 P eed7c39853c04344aacc55afdd26f1fe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e4f629e966e419fb21b57161abdf46c in term 1.
I20260619 10:17:37.992769 8677 leader_election.cc:302] T 3911676e1c43437489d614d1c544ec86 P 0e4f629e966e419fb21b57161abdf46c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0c4c7226ac8e48ca8d708ea8bf3b76f7, 0e4f629e966e419fb21b57161abdf46c; no voters:
I20260619 10:17:37.992882 8821 raft_consensus.cc:2781] T 3911676e1c43437489d614d1c544ec86 P 0e4f629e966e419fb21b57161abdf46c [term 1 FOLLOWER]: Leader election won for term 1
I20260619 10:17:37.992938 8821 raft_consensus.cc:686] T 3911676e1c43437489d614d1c544ec86 P 0e4f629e966e419fb21b57161abdf46c [term 1 LEADER]: Becoming Leader. State: Replica: 0e4f629e966e419fb21b57161abdf46c, State: Running, Role: LEADER
I20260619 10:17:37.993005 8821 consensus_queue.cc:237] T 3911676e1c43437489d614d1c544ec86 P 0e4f629e966e419fb21b57161abdf46c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c4c7226ac8e48ca8d708ea8bf3b76f7" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 37567 } } peers { permanent_uuid: "eed7c39853c04344aacc55afdd26f1fe" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 36277 } } peers { permanent_uuid: "0e4f629e966e419fb21b57161abdf46c" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 42531 } }
I20260619 10:17:37.993712 8307 catalog_manager.cc:5495] T 3911676e1c43437489d614d1c544ec86 P 0e4f629e966e419fb21b57161abdf46c reported cstate change: term changed from 0 to 1, leader changed from <none> to 0e4f629e966e419fb21b57161abdf46c (127.0.206.67). New cstate: current_term: 1 leader_uuid: "0e4f629e966e419fb21b57161abdf46c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c4c7226ac8e48ca8d708ea8bf3b76f7" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 37567 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eed7c39853c04344aacc55afdd26f1fe" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 36277 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0e4f629e966e419fb21b57161abdf46c" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 42531 } health_report { overall_health: HEALTHY } } }
I20260619 10:17:38.014266 8893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:38.011996 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36844 (local address 127.0.206.67:42531)
0619 10:17:38.012146 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0619 10:17:38.012148 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 10:17:38.012155 (+ 7us) server_negotiation.cc:407] Connection header received
0619 10:17:38.012187 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:17:38.012187 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:17:38.012204 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:17:38.012274 (+ 70us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 10:17:38.012377 (+ 103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:38.012919 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:38.013615 (+ 696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:38.013797 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:38.013845 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:38.013992 (+ 147us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 10:17:38.014093 (+ 101us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 10:17:38.014144 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0619 10:17:38.014187 (+ 43us) server_negotiation.cc:299] Negotiation successful
0619 10:17:38.014202 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":120,"thread_start_us":101,"threads_started":1}
I20260619 10:17:38.015157 8464 raft_consensus.cc:1261] T 3911676e1c43437489d614d1c544ec86 P 0c4c7226ac8e48ca8d708ea8bf3b76f7 [term 1 FOLLOWER]: Refusing update from remote peer 0e4f629e966e419fb21b57161abdf46c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 10:17:38.015172 8604 raft_consensus.cc:1261] T 3911676e1c43437489d614d1c544ec86 P eed7c39853c04344aacc55afdd26f1fe [term 1 FOLLOWER]: Refusing update from remote peer 0e4f629e966e419fb21b57161abdf46c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 10:17:38.015383 8821 consensus_queue.cc:1035] T 3911676e1c43437489d614d1c544ec86 P 0e4f629e966e419fb21b57161abdf46c [LEADER]: Connected to new peer: Peer: permanent_uuid: "0c4c7226ac8e48ca8d708ea8bf3b76f7" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 37567 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 10:17:38.015497 8861 consensus_queue.cc:1035] T 3911676e1c43437489d614d1c544ec86 P 0e4f629e966e419fb21b57161abdf46c [LEADER]: Connected to new peer: Peer: permanent_uuid: "eed7c39853c04344aacc55afdd26f1fe" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 36277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 10:17:38.019255 8306 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58442:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 10:17:38.019345 8306 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58442:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 10:17:38.019928 8306 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f99ceb70a6684009b753712e007c5de5: Sending DeleteTablet for 3 replicas of tablet 3911676e1c43437489d614d1c544ec86
I20260619 10:17:38.020159 8444 tablet_service.cc:1514] Processing DeleteTablet for tablet 3911676e1c43437489d614d1c544ec86 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 10:17:38 UTC) from {username='oryx', principal='oryx/127.0.206.126@KRBTEST.COM'} at 127.0.0.1:43058
I20260619 10:17:38.020177 8584 tablet_service.cc:1514] Processing DeleteTablet for tablet 3911676e1c43437489d614d1c544ec86 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 10:17:38 UTC) from {username='oryx', principal='oryx/127.0.206.126@KRBTEST.COM'} at 127.0.0.1:36298
I20260619 10:17:38.020228 8724 tablet_service.cc:1514] Processing DeleteTablet for tablet 3911676e1c43437489d614d1c544ec86 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 10:17:38 UTC) from {username='oryx', principal='oryx/127.0.206.126@KRBTEST.COM'} at 127.0.0.1:36834
I20260619 10:17:38.020248 8838 tablet_replica.cc:330] T 3911676e1c43437489d614d1c544ec86 P 0c4c7226ac8e48ca8d708ea8bf3b76f7: stopping tablet replica
I20260619 10:17:38.020267 8839 tablet_replica.cc:330] T 3911676e1c43437489d614d1c544ec86 P eed7c39853c04344aacc55afdd26f1fe: stopping tablet replica
I20260619 10:17:38.020303 8838 raft_consensus.cc:2229] T 3911676e1c43437489d614d1c544ec86 P 0c4c7226ac8e48ca8d708ea8bf3b76f7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 10:17:38.020308 8839 raft_consensus.cc:2229] T 3911676e1c43437489d614d1c544ec86 P eed7c39853c04344aacc55afdd26f1fe [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 10:17:38.020344 8838 raft_consensus.cc:2258] T 3911676e1c43437489d614d1c544ec86 P 0c4c7226ac8e48ca8d708ea8bf3b76f7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 10:17:38.020349 8839 raft_consensus.cc:2258] T 3911676e1c43437489d614d1c544ec86 P eed7c39853c04344aacc55afdd26f1fe [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 10:17:38.020434 8840 tablet_replica.cc:330] T 3911676e1c43437489d614d1c544ec86 P 0e4f629e966e419fb21b57161abdf46c: stopping tablet replica
I20260619 10:17:38.020484 8840 raft_consensus.cc:2229] T 3911676e1c43437489d614d1c544ec86 P 0e4f629e966e419fb21b57161abdf46c [term 1 LEADER]: Raft consensus shutting down.
I20260619 10:17:38.020561 8840 raft_consensus.cc:2258] T 3911676e1c43437489d614d1c544ec86 P 0e4f629e966e419fb21b57161abdf46c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 10:17:38.020875 8838 ts_tablet_manager.cc:1905] T 3911676e1c43437489d614d1c544ec86 P 0c4c7226ac8e48ca8d708ea8bf3b76f7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 10:17:38.020972 8840 ts_tablet_manager.cc:1905] T 3911676e1c43437489d614d1c544ec86 P 0e4f629e966e419fb21b57161abdf46c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 10:17:38.020983 8839 ts_tablet_manager.cc:1905] T 3911676e1c43437489d614d1c544ec86 P eed7c39853c04344aacc55afdd26f1fe: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 10:17:38.022452 8840 ts_tablet_manager.cc:1918] T 3911676e1c43437489d614d1c544ec86 P 0e4f629e966e419fb21b57161abdf46c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 10:17:38.022400 8306 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58442:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260619 10:17:38.022495 8840 log.cc:1196] T 3911676e1c43437489d614d1c544ec86 P 0e4f629e966e419fb21b57161abdf46c: Deleting WAL directory at /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781864224577058-825-0/minicluster-data/ts-2/wal/wals/3911676e1c43437489d614d1c544ec86
I20260619 10:17:38.022543 8839 ts_tablet_manager.cc:1918] T 3911676e1c43437489d614d1c544ec86 P eed7c39853c04344aacc55afdd26f1fe: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
W20260619 10:17:38.022558 8306 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 10:17:38.022755 8840 ts_tablet_manager.cc:1939] T 3911676e1c43437489d614d1c544ec86 P 0e4f629e966e419fb21b57161abdf46c: Deleting consensus metadata
I20260619 10:17:38.022886 8838 ts_tablet_manager.cc:1918] T 3911676e1c43437489d614d1c544ec86 P 0c4c7226ac8e48ca8d708ea8bf3b76f7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 10:17:38.022927 8838 log.cc:1196] T 3911676e1c43437489d614d1c544ec86 P 0c4c7226ac8e48ca8d708ea8bf3b76f7: Deleting WAL directory at /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781864224577058-825-0/minicluster-data/ts-0/wal/wals/3911676e1c43437489d614d1c544ec86
I20260619 10:17:38.023125 8838 ts_tablet_manager.cc:1939] T 3911676e1c43437489d614d1c544ec86 P 0c4c7226ac8e48ca8d708ea8bf3b76f7: Deleting consensus metadata
I20260619 10:17:38.023150 8839 log.cc:1196] T 3911676e1c43437489d614d1c544ec86 P eed7c39853c04344aacc55afdd26f1fe: Deleting WAL directory at /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781864224577058-825-0/minicluster-data/ts-1/wal/wals/3911676e1c43437489d614d1c544ec86
I20260619 10:17:38.023398 8839 ts_tablet_manager.cc:1939] T 3911676e1c43437489d614d1c544ec86 P eed7c39853c04344aacc55afdd26f1fe: Deleting consensus metadata
I20260619 10:17:38.023403 8294 catalog_manager.cc:4841] TS 0e4f629e966e419fb21b57161abdf46c (127.0.206.67:42531): tablet 3911676e1c43437489d614d1c544ec86 (table test-table [id=96e3553b61a24cfeb0addbe4e573902d]) successfully deleted
I20260619 10:17:38.023722 8293 catalog_manager.cc:4841] TS 0c4c7226ac8e48ca8d708ea8bf3b76f7 (127.0.206.65:37567): tablet 3911676e1c43437489d614d1c544ec86 (table test-table [id=96e3553b61a24cfeb0addbe4e573902d]) successfully deleted
I20260619 10:17:38.023861 8294 catalog_manager.cc:4841] TS eed7c39853c04344aacc55afdd26f1fe (127.0.206.66:36277): tablet 3911676e1c43437489d614d1c544ec86 (table test-table [id=96e3553b61a24cfeb0addbe4e573902d]) successfully deleted
I20260619 10:17:38.024294 8444 tablet_service.cc:1467] Processing CreateTablet for tablet b60425b2279d47f4a581475f1f6db709 (DEFAULT_TABLE table=test-table [id=f19e41556e394d21a283a638889af696]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 10:17:38.024322 8584 tablet_service.cc:1467] Processing CreateTablet for tablet b60425b2279d47f4a581475f1f6db709 (DEFAULT_TABLE table=test-table [id=f19e41556e394d21a283a638889af696]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 10:17:38.024374 8724 tablet_service.cc:1467] Processing CreateTablet for tablet b60425b2279d47f4a581475f1f6db709 (DEFAULT_TABLE table=test-table [id=f19e41556e394d21a283a638889af696]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 10:17:38.024402 8444 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b60425b2279d47f4a581475f1f6db709. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 10:17:38.024420 8584 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b60425b2279d47f4a581475f1f6db709. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 10:17:38.024452 8724 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b60425b2279d47f4a581475f1f6db709. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 10:17:38.025571 8816 tablet_bootstrap.cc:492] T b60425b2279d47f4a581475f1f6db709 P 0e4f629e966e419fb21b57161abdf46c: Bootstrap starting.
I20260619 10:17:38.025599 8820 tablet_bootstrap.cc:492] T b60425b2279d47f4a581475f1f6db709 P eed7c39853c04344aacc55afdd26f1fe: Bootstrap starting.
I20260619 10:17:38.025895 8816 tablet_bootstrap.cc:654] T b60425b2279d47f4a581475f1f6db709 P 0e4f629e966e419fb21b57161abdf46c: Neither blocks nor log segments found. Creating new log.
I20260619 10:17:38.025903 8820 tablet_bootstrap.cc:654] T b60425b2279d47f4a581475f1f6db709 P eed7c39853c04344aacc55afdd26f1fe: Neither blocks nor log segments found. Creating new log.
I20260619 10:17:38.026090 8815 tablet_bootstrap.cc:492] T b60425b2279d47f4a581475f1f6db709 P 0c4c7226ac8e48ca8d708ea8bf3b76f7: Bootstrap starting.
I20260619 10:17:38.026362 8815 tablet_bootstrap.cc:654] T b60425b2279d47f4a581475f1f6db709 P 0c4c7226ac8e48ca8d708ea8bf3b76f7: Neither blocks nor log segments found. Creating new log.
I20260619 10:17:38.026443 8816 tablet_bootstrap.cc:492] T b60425b2279d47f4a581475f1f6db709 P 0e4f629e966e419fb21b57161abdf46c: No bootstrap required, opened a new log
I20260619 10:17:38.026472 8816 ts_tablet_manager.cc:1397] T b60425b2279d47f4a581475f1f6db709 P 0e4f629e966e419fb21b57161abdf46c: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20260619 10:17:38.026633 8816 raft_consensus.cc:348] T b60425b2279d47f4a581475f1f6db709 P 0e4f629e966e419fb21b57161abdf46c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c4c7226ac8e48ca8d708ea8bf3b76f7" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 37567 } } peers { permanent_uuid: "eed7c39853c04344aacc55afdd26f1fe" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 36277 } } peers { permanent_uuid: "0e4f629e966e419fb21b57161abdf46c" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 42531 } }
I20260619 10:17:38.026708 8816 raft_consensus.cc:374] T b60425b2279d47f4a581475f1f6db709 P 0e4f629e966e419fb21b57161abdf46c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 10:17:38.026734 8816 raft_consensus.cc:729] T b60425b2279d47f4a581475f1f6db709 P 0e4f629e966e419fb21b57161abdf46c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e4f629e966e419fb21b57161abdf46c, State: Initialized, Role: FOLLOWER
I20260619 10:17:38.026791 8816 consensus_queue.cc:260] T b60425b2279d47f4a581475f1f6db709 P 0e4f629e966e419fb21b57161abdf46c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c4c7226ac8e48ca8d708ea8bf3b76f7" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 37567 } } peers { permanent_uuid: "eed7c39853c04344aacc55afdd26f1fe" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 36277 } } peers { permanent_uuid: "0e4f629e966e419fb21b57161abdf46c" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 42531 } }
I20260619 10:17:38.026885 8816 ts_tablet_manager.cc:1428] T b60425b2279d47f4a581475f1f6db709 P 0e4f629e966e419fb21b57161abdf46c: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260619 10:17:38.027011 8815 tablet_bootstrap.cc:492] T b60425b2279d47f4a581475f1f6db709 P 0c4c7226ac8e48ca8d708ea8bf3b76f7: No bootstrap required, opened a new log
I20260619 10:17:38.027055 8815 ts_tablet_manager.cc:1397] T b60425b2279d47f4a581475f1f6db709 P 0c4c7226ac8e48ca8d708ea8bf3b76f7: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260619 10:17:38.027148 8815 raft_consensus.cc:348] T b60425b2279d47f4a581475f1f6db709 P 0c4c7226ac8e48ca8d708ea8bf3b76f7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c4c7226ac8e48ca8d708ea8bf3b76f7" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 37567 } } peers { permanent_uuid: "eed7c39853c04344aacc55afdd26f1fe" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 36277 } } peers { permanent_uuid: "0e4f629e966e419fb21b57161abdf46c" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 42531 } }
I20260619 10:17:38.027207 8815 raft_consensus.cc:374] T b60425b2279d47f4a581475f1f6db709 P 0c4c7226ac8e48ca8d708ea8bf3b76f7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 10:17:38.027227 8815 raft_consensus.cc:729] T b60425b2279d47f4a581475f1f6db709 P 0c4c7226ac8e48ca8d708ea8bf3b76f7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0c4c7226ac8e48ca8d708ea8bf3b76f7, State: Initialized, Role: FOLLOWER
I20260619 10:17:38.027274 8815 consensus_queue.cc:260] T b60425b2279d47f4a581475f1f6db709 P 0c4c7226ac8e48ca8d708ea8bf3b76f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c4c7226ac8e48ca8d708ea8bf3b76f7" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 37567 } } peers { permanent_uuid: "eed7c39853c04344aacc55afdd26f1fe" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 36277 } } peers { permanent_uuid: "0e4f629e966e419fb21b57161abdf46c" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 42531 } }
I20260619 10:17:38.027359 8815 ts_tablet_manager.cc:1428] T b60425b2279d47f4a581475f1f6db709 P 0c4c7226ac8e48ca8d708ea8bf3b76f7: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260619 10:17:38.028306 8820 tablet_bootstrap.cc:492] T b60425b2279d47f4a581475f1f6db709 P eed7c39853c04344aacc55afdd26f1fe: No bootstrap required, opened a new log
I20260619 10:17:38.028350 8820 ts_tablet_manager.cc:1397] T b60425b2279d47f4a581475f1f6db709 P eed7c39853c04344aacc55afdd26f1fe: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260619 10:17:38.028439 8820 raft_consensus.cc:348] T b60425b2279d47f4a581475f1f6db709 P eed7c39853c04344aacc55afdd26f1fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c4c7226ac8e48ca8d708ea8bf3b76f7" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 37567 } } peers { permanent_uuid: "eed7c39853c04344aacc55afdd26f1fe" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 36277 } } peers { permanent_uuid: "0e4f629e966e419fb21b57161abdf46c" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 42531 } }
I20260619 10:17:38.028508 8820 raft_consensus.cc:374] T b60425b2279d47f4a581475f1f6db709 P eed7c39853c04344aacc55afdd26f1fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 10:17:38.028527 8820 raft_consensus.cc:729] T b60425b2279d47f4a581475f1f6db709 P eed7c39853c04344aacc55afdd26f1fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eed7c39853c04344aacc55afdd26f1fe, State: Initialized, Role: FOLLOWER
I20260619 10:17:38.028569 8820 consensus_queue.cc:260] T b60425b2279d47f4a581475f1f6db709 P eed7c39853c04344aacc55afdd26f1fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c4c7226ac8e48ca8d708ea8bf3b76f7" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 37567 } } peers { permanent_uuid: "eed7c39853c04344aacc55afdd26f1fe" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 36277 } } peers { permanent_uuid: "0e4f629e966e419fb21b57161abdf46c" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 42531 } }
I20260619 10:17:38.028661 8820 ts_tablet_manager.cc:1428] T b60425b2279d47f4a581475f1f6db709 P eed7c39853c04344aacc55afdd26f1fe: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260619 10:17:38.050324 8819 raft_consensus.cc:482] T b60425b2279d47f4a581475f1f6db709 P 0c4c7226ac8e48ca8d708ea8bf3b76f7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 10:17:38.050413 8819 raft_consensus.cc:504] T b60425b2279d47f4a581475f1f6db709 P 0c4c7226ac8e48ca8d708ea8bf3b76f7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c4c7226ac8e48ca8d708ea8bf3b76f7" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 37567 } } peers { permanent_uuid: "eed7c39853c04344aacc55afdd26f1fe" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 36277 } } peers { permanent_uuid: "0e4f629e966e419fb21b57161abdf46c" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 42531 } }
I20260619 10:17:38.050554 8819 leader_election.cc:288] T b60425b2279d47f4a581475f1f6db709 P 0c4c7226ac8e48ca8d708ea8bf3b76f7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eed7c39853c04344aacc55afdd26f1fe (127.0.206.66:36277), 0e4f629e966e419fb21b57161abdf46c (127.0.206.67:42531)
I20260619 10:17:38.050793 8604 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b60425b2279d47f4a581475f1f6db709" candidate_uuid: "0c4c7226ac8e48ca8d708ea8bf3b76f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eed7c39853c04344aacc55afdd26f1fe" is_pre_election: true
I20260619 10:17:38.050819 8744 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b60425b2279d47f4a581475f1f6db709" candidate_uuid: "0c4c7226ac8e48ca8d708ea8bf3b76f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e4f629e966e419fb21b57161abdf46c" is_pre_election: true
I20260619 10:17:38.050899 8604 raft_consensus.cc:2454] T b60425b2279d47f4a581475f1f6db709 P eed7c39853c04344aacc55afdd26f1fe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0c4c7226ac8e48ca8d708ea8bf3b76f7 in term 0.
I20260619 10:17:38.050917 8744 raft_consensus.cc:2454] T b60425b2279d47f4a581475f1f6db709 P 0e4f629e966e419fb21b57161abdf46c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0c4c7226ac8e48ca8d708ea8bf3b76f7 in term 0.
I20260619 10:17:38.051045 8398 leader_election.cc:302] T b60425b2279d47f4a581475f1f6db709 P 0c4c7226ac8e48ca8d708ea8bf3b76f7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0c4c7226ac8e48ca8d708ea8bf3b76f7, eed7c39853c04344aacc55afdd26f1fe; no voters:
I20260619 10:17:38.051174 8819 raft_consensus.cc:2781] T b60425b2279d47f4a581475f1f6db709 P 0c4c7226ac8e48ca8d708ea8bf3b76f7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 10:17:38.051211 8819 raft_consensus.cc:482] T b60425b2279d47f4a581475f1f6db709 P 0c4c7226ac8e48ca8d708ea8bf3b76f7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 10:17:38.051231 8819 raft_consensus.cc:3037] T b60425b2279d47f4a581475f1f6db709 P 0c4c7226ac8e48ca8d708ea8bf3b76f7 [term 0 FOLLOWER]: Advancing to term 1
I20260619 10:17:38.052052 8819 raft_consensus.cc:504] T b60425b2279d47f4a581475f1f6db709 P 0c4c7226ac8e48ca8d708ea8bf3b76f7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c4c7226ac8e48ca8d708ea8bf3b76f7" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 37567 } } peers { permanent_uuid: "eed7c39853c04344aacc55afdd26f1fe" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 36277 } } peers { permanent_uuid: "0e4f629e966e419fb21b57161abdf46c" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 42531 } }
I20260619 10:17:38.052175 8819 leader_election.cc:288] T b60425b2279d47f4a581475f1f6db709 P 0c4c7226ac8e48ca8d708ea8bf3b76f7 [CANDIDATE]: Term 1 election: Requested vote from peers eed7c39853c04344aacc55afdd26f1fe (127.0.206.66:36277), 0e4f629e966e419fb21b57161abdf46c (127.0.206.67:42531)
I20260619 10:17:38.052343 8604 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b60425b2279d47f4a581475f1f6db709" candidate_uuid: "0c4c7226ac8e48ca8d708ea8bf3b76f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eed7c39853c04344aacc55afdd26f1fe"
I20260619 10:17:38.052366 8744 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b60425b2279d47f4a581475f1f6db709" candidate_uuid: "0c4c7226ac8e48ca8d708ea8bf3b76f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e4f629e966e419fb21b57161abdf46c"
I20260619 10:17:38.052433 8744 raft_consensus.cc:3037] T b60425b2279d47f4a581475f1f6db709 P 0e4f629e966e419fb21b57161abdf46c [term 0 FOLLOWER]: Advancing to term 1
I20260619 10:17:38.052439 8604 raft_consensus.cc:3037] T b60425b2279d47f4a581475f1f6db709 P eed7c39853c04344aacc55afdd26f1fe [term 0 FOLLOWER]: Advancing to term 1
I20260619 10:17:38.053141 8744 raft_consensus.cc:2454] T b60425b2279d47f4a581475f1f6db709 P 0e4f629e966e419fb21b57161abdf46c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0c4c7226ac8e48ca8d708ea8bf3b76f7 in term 1.
I20260619 10:17:38.053158 8604 raft_consensus.cc:2454] T b60425b2279d47f4a581475f1f6db709 P eed7c39853c04344aacc55afdd26f1fe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0c4c7226ac8e48ca8d708ea8bf3b76f7 in term 1.
I20260619 10:17:38.053308 8398 leader_election.cc:302] T b60425b2279d47f4a581475f1f6db709 P 0c4c7226ac8e48ca8d708ea8bf3b76f7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0c4c7226ac8e48ca8d708ea8bf3b76f7, 0e4f629e966e419fb21b57161abdf46c; no voters:
I20260619 10:17:38.053423 8819 raft_consensus.cc:2781] T b60425b2279d47f4a581475f1f6db709 P 0c4c7226ac8e48ca8d708ea8bf3b76f7 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 10:17:38.053478 8819 raft_consensus.cc:686] T b60425b2279d47f4a581475f1f6db709 P 0c4c7226ac8e48ca8d708ea8bf3b76f7 [term 1 LEADER]: Becoming Leader. State: Replica: 0c4c7226ac8e48ca8d708ea8bf3b76f7, State: Running, Role: LEADER
I20260619 10:17:38.053522 8819 consensus_queue.cc:237] T b60425b2279d47f4a581475f1f6db709 P 0c4c7226ac8e48ca8d708ea8bf3b76f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c4c7226ac8e48ca8d708ea8bf3b76f7" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 37567 } } peers { permanent_uuid: "eed7c39853c04344aacc55afdd26f1fe" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 36277 } } peers { permanent_uuid: "0e4f629e966e419fb21b57161abdf46c" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 42531 } }
I20260619 10:17:38.054097 8306 catalog_manager.cc:5495] T b60425b2279d47f4a581475f1f6db709 P 0c4c7226ac8e48ca8d708ea8bf3b76f7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0c4c7226ac8e48ca8d708ea8bf3b76f7 (127.0.206.65). New cstate: current_term: 1 leader_uuid: "0c4c7226ac8e48ca8d708ea8bf3b76f7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c4c7226ac8e48ca8d708ea8bf3b76f7" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 37567 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "eed7c39853c04344aacc55afdd26f1fe" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 36277 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0e4f629e966e419fb21b57161abdf46c" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 42531 } health_report { overall_health: UNKNOWN } } }
I20260619 10:17:38.067664 8813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:38.065238 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43078 (local address 127.0.206.65:37567)
0619 10:17:38.065294 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0619 10:17:38.065295 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 10:17:38.065302 (+ 7us) server_negotiation.cc:407] Connection header received
0619 10:17:38.065320 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:17:38.065320 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:17:38.065329 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:17:38.065375 (+ 46us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 10:17:38.065513 (+ 138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:38.065964 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:38.067063 (+ 1099us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:38.067242 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:38.067295 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:38.067442 (+ 147us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 10:17:38.067466 (+ 24us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 10:17:38.067499 (+ 33us) server_negotiation.cc:1035] Waiting for connection context
0619 10:17:38.067565 (+ 66us) server_negotiation.cc:299] Negotiation successful
0619 10:17:38.067580 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260619 10:17:38.071857 8813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:38.069729 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.65:36485 (local address 127.0.206.65:37567)
0619 10:17:38.069780 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0619 10:17:38.069782 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 10:17:38.069791 (+ 9us) server_negotiation.cc:407] Connection header received
0619 10:17:38.069847 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:17:38.069848 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:17:38.069857 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:17:38.069909 (+ 52us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 10:17:38.070001 (+ 92us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:38.070441 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:38.071215 (+ 774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:38.071412 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:38.071484 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:38.071629 (+ 145us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 10:17:38.071657 (+ 28us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 10:17:38.071703 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0619 10:17:38.071757 (+ 54us) server_negotiation.cc:299] Negotiation successful
0619 10:17:38.071775 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":35}
I20260619 10:17:38.071943 8870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:38.069606 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.65:37567 (local address 127.0.206.65:36485)
0619 10:17:38.069727 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0619 10:17:38.069735 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0619 10:17:38.069790 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 10:17:38.069912 (+ 122us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 10:17:38.069912 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 10:17:38.069913 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0619 10:17:38.069956 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:38.069957 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:38.070523 (+ 566us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:38.070524 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:38.071123 (+ 599us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:38.071125 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:38.071480 (+ 355us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:38.071481 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:38.071527 (+ 46us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:38.071532 (+ 5us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0619 10:17:38.071702 (+ 170us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0619 10:17:38.071703 (+ 1us) client_negotiation.cc:769] Sending connection context
0619 10:17:38.071753 (+ 50us) client_negotiation.cc:240] Negotiation successful
0619 10:17:38.071813 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":103,"mutex_wait_us":2}
I20260619 10:17:38.074188 8604 raft_consensus.cc:1261] T b60425b2279d47f4a581475f1f6db709 P eed7c39853c04344aacc55afdd26f1fe [term 1 FOLLOWER]: Refusing update from remote peer 0c4c7226ac8e48ca8d708ea8bf3b76f7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 10:17:38.074206 8744 raft_consensus.cc:1261] T b60425b2279d47f4a581475f1f6db709 P 0e4f629e966e419fb21b57161abdf46c [term 1 FOLLOWER]: Refusing update from remote peer 0c4c7226ac8e48ca8d708ea8bf3b76f7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 10:17:38.074388 8833 consensus_queue.cc:1035] T b60425b2279d47f4a581475f1f6db709 P 0c4c7226ac8e48ca8d708ea8bf3b76f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e4f629e966e419fb21b57161abdf46c" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 42531 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 10:17:38.074476 8825 consensus_queue.cc:1035] T b60425b2279d47f4a581475f1f6db709 P 0c4c7226ac8e48ca8d708ea8bf3b76f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eed7c39853c04344aacc55afdd26f1fe" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 36277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 10:17:38.084986 8306 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58442:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 10:17:38.085095 8306 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58442:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 10:17:38.085677 8306 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f99ceb70a6684009b753712e007c5de5: Sending DeleteTablet for 3 replicas of tablet b60425b2279d47f4a581475f1f6db709
I20260619 10:17:38.085942 8444 tablet_service.cc:1514] Processing DeleteTablet for tablet b60425b2279d47f4a581475f1f6db709 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 10:17:38 UTC) from {username='oryx', principal='oryx/127.0.206.126@KRBTEST.COM'} at 127.0.0.1:43058
I20260619 10:17:38.085961 8584 tablet_service.cc:1514] Processing DeleteTablet for tablet b60425b2279d47f4a581475f1f6db709 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 10:17:38 UTC) from {username='oryx', principal='oryx/127.0.206.126@KRBTEST.COM'} at 127.0.0.1:36298
I20260619 10:17:38.086041 8724 tablet_service.cc:1514] Processing DeleteTablet for tablet b60425b2279d47f4a581475f1f6db709 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 10:17:38 UTC) from {username='oryx', principal='oryx/127.0.206.126@KRBTEST.COM'} at 127.0.0.1:36834
I20260619 10:17:38.086104 8839 tablet_replica.cc:330] T b60425b2279d47f4a581475f1f6db709 P eed7c39853c04344aacc55afdd26f1fe: stopping tablet replica
I20260619 10:17:38.086144 8838 tablet_replica.cc:330] T b60425b2279d47f4a581475f1f6db709 P 0c4c7226ac8e48ca8d708ea8bf3b76f7: stopping tablet replica
I20260619 10:17:38.086148 8840 tablet_replica.cc:330] T b60425b2279d47f4a581475f1f6db709 P 0e4f629e966e419fb21b57161abdf46c: stopping tablet replica
I20260619 10:17:38.086165 8839 raft_consensus.cc:2229] T b60425b2279d47f4a581475f1f6db709 P eed7c39853c04344aacc55afdd26f1fe [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 10:17:38.086192 8840 raft_consensus.cc:2229] T b60425b2279d47f4a581475f1f6db709 P 0e4f629e966e419fb21b57161abdf46c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 10:17:38.086197 8838 raft_consensus.cc:2229] T b60425b2279d47f4a581475f1f6db709 P 0c4c7226ac8e48ca8d708ea8bf3b76f7 [term 1 LEADER]: Raft consensus shutting down.
I20260619 10:17:38.086210 8839 raft_consensus.cc:2258] T b60425b2279d47f4a581475f1f6db709 P eed7c39853c04344aacc55afdd26f1fe [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 10:17:38.086225 8840 raft_consensus.cc:2258] T b60425b2279d47f4a581475f1f6db709 P 0e4f629e966e419fb21b57161abdf46c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 10:17:38.086249 8838 raft_consensus.cc:2258] T b60425b2279d47f4a581475f1f6db709 P 0c4c7226ac8e48ca8d708ea8bf3b76f7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 10:17:38.086555 8838 ts_tablet_manager.cc:1905] T b60425b2279d47f4a581475f1f6db709 P 0c4c7226ac8e48ca8d708ea8bf3b76f7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 10:17:38.086555 8840 ts_tablet_manager.cc:1905] T b60425b2279d47f4a581475f1f6db709 P 0e4f629e966e419fb21b57161abdf46c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 10:17:38.086700 825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTU9OP6/build/release/bin/kudu with pid 8379
I20260619 10:17:38.088133 8840 ts_tablet_manager.cc:1918] T b60425b2279d47f4a581475f1f6db709 P 0e4f629e966e419fb21b57161abdf46c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260619 10:17:38.088184 8840 log.cc:1196] T b60425b2279d47f4a581475f1f6db709 P 0e4f629e966e419fb21b57161abdf46c: Deleting WAL directory at /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781864224577058-825-0/minicluster-data/ts-2/wal/wals/b60425b2279d47f4a581475f1f6db709
I20260619 10:17:38.088433 8840 ts_tablet_manager.cc:1939] T b60425b2279d47f4a581475f1f6db709 P 0e4f629e966e419fb21b57161abdf46c: Deleting consensus metadata
I20260619 10:17:38.088737 8294 catalog_manager.cc:4841] TS 0e4f629e966e419fb21b57161abdf46c (127.0.206.67:42531): tablet b60425b2279d47f4a581475f1f6db709 (table test-table [id=f19e41556e394d21a283a638889af696]) successfully deleted
I20260619 10:17:38.090340 8839 ts_tablet_manager.cc:1905] T b60425b2279d47f4a581475f1f6db709 P eed7c39853c04344aacc55afdd26f1fe: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 10:17:38.091564 8839 ts_tablet_manager.cc:1918] T b60425b2279d47f4a581475f1f6db709 P eed7c39853c04344aacc55afdd26f1fe: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260619 10:17:38.091614 8839 log.cc:1196] T b60425b2279d47f4a581475f1f6db709 P eed7c39853c04344aacc55afdd26f1fe: Deleting WAL directory at /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781864224577058-825-0/minicluster-data/ts-1/wal/wals/b60425b2279d47f4a581475f1f6db709
I20260619 10:17:38.091766 8839 ts_tablet_manager.cc:1939] T b60425b2279d47f4a581475f1f6db709 P eed7c39853c04344aacc55afdd26f1fe: Deleting consensus metadata
I20260619 10:17:38.092059 8294 catalog_manager.cc:4841] TS eed7c39853c04344aacc55afdd26f1fe (127.0.206.66:36277): tablet b60425b2279d47f4a581475f1f6db709 (table test-table [id=f19e41556e394d21a283a638889af696]) successfully deleted
W20260619 10:17:38.095733 8293 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.0.206.65:37567 (error 108)
I20260619 10:17:38.095971 825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTU9OP6/build/release/bin/kudu with pid 8519
I20260619 10:17:38.096691 8906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:38.096074 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.65:37567 (local address 127.0.0.1:43080)
0619 10:17:38.096628 (+ 554us) negotiation.cc:107] Waiting for socket to connect
0619 10:17:38.096661 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.65:37567: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":541,"thread_start_us":34,"threads_started":1}
W20260619 10:17:38.096791 8293 catalog_manager.cc:4584] TS 0c4c7226ac8e48ca8d708ea8bf3b76f7 (127.0.206.65:37567): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet b60425b2279d47f4a581475f1f6db709: Network error: Client connection negotiation failed: client connection to 127.0.206.65:37567: connect: Connection refused (error 111)
I20260619 10:17:38.104805 825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTU9OP6/build/release/bin/kudu with pid 8659
I20260619 10:17:38.109640 825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTU9OP6/build/release/bin/kudu with pid 8275
2026-06-19T10:17:38Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (2720 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781864224577058-825-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 10:17:38 dist-test-slave-6pt2 krb5kdc[8909](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 10:17:38 dist-test-slave-6pt2 krb5kdc[8909](info): set up 2 sockets
Jun 19 10:17:38 dist-test-slave-6pt2 krb5kdc[8909](info): commencing operation
krb5kdc: starting...
W20260619 10:17:39.143100 825 mini_kdc.cc:121] Time spent starting KDC: real 1.013s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781864224577058-825-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781864224577058-825-0/krb5kdc/test-user.keytab.
Jun 19 10:17:39 dist-test-slave-6pt2 krb5kdc[8909](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864259, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T10:17:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T10:17:39Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.206.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.206.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.206.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781864224577058-825-0/krb5kdc/kudu_127.0.206.126.keytab.
Entry for principal kudu/127.0.206.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781864224577058-825-0/krb5kdc/kudu_127.0.206.126.keytab.
I20260619 10:17:39.251221 825 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTU9OP6/build/release/bin/kudu
/tmp/dist-test-taskTU9OP6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781864224577058-825-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781864224577058-825-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781864224577058-825-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781864224577058-825-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.206.126:38001
--webserver_interface=127.0.206.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781864224577058-825-0/krb5kdc/kudu_127.0.206.126.keytab
--principal=kudu/127.0.206.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.206.84:41983
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.206.126:38001
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781864224577058-825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781864224577058-825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781864224577058-825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 10:17:39.327178 8922 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 10:17:39.327376 8922 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 10:17:39.327399 8922 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 10:17:39.328411 8922 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 10:17:39.328442 8922 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 10:17:39.328456 8922 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 10:17:39.328469 8922 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 10:17:39.328480 8922 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 10:17:39.329767 8922 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.206.84:41983
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781864224577058-825-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781864224577058-825-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.206.126:38001
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781864224577058-825-0/krb5kdc/kudu_127.0.206.126.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.206.126:38001
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781864224577058-825-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.206.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8922
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781864224577058-825-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 09:32:44 UTC on ccycloud.quasar-gbtldp.root.comops.site
build id 4455440
I20260619 10:17:39.329923 8922 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 10:17:39.330113 8922 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 10:17:39.334781 8930 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 10:17:39.334800 8928 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 10:17:39.334937 8922 server_base.cc:1034] running on GCE node
W20260619 10:17:39.334980 8927 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 10:17:39.335327 8922 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 10:17:39.335595 8922 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 10:17:39.336740 8922 hybrid_clock.cc:648] HybridClock initialized: now 1781864259336728 us; error 29 us; skew 500 ppm
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-06-19T10:17:39Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (1227 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781864224577058-825-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 10:17:39 dist-test-slave-6pt2 krb5kdc[8935](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 10:17:39 dist-test-slave-6pt2 krb5kdc[8935](info): set up 2 sockets
Jun 19 10:17:39 dist-test-slave-6pt2 krb5kdc[8935](info): commencing operation
krb5kdc: starting...
W20260619 10:17:40.350953 825 mini_kdc.cc:121] Time spent starting KDC: real 0.993s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781864224577058-825-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781864224577058-825-0/krb5kdc/test-user.keytab.
Jun 19 10:17:40 dist-test-slave-6pt2 krb5kdc[8935](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864260, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T10:17:40Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T10:17:40Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.206.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.206.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.206.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781864224577058-825-0/krb5kdc/kudu_127.0.206.126.keytab.
Entry for principal kudu/127.0.206.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781864224577058-825-0/krb5kdc/kudu_127.0.206.126.keytab.
I20260619 10:17:40.459196 825 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTU9OP6/build/release/bin/kudu
/tmp/dist-test-taskTU9OP6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781864224577058-825-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781864224577058-825-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781864224577058-825-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781864224577058-825-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.206.126:45539
--webserver_interface=127.0.206.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781864224577058-825-0/krb5kdc/kudu_127.0.206.126.keytab
--principal=kudu/127.0.206.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.206.84:38663
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.206.126:45539
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781864224577058-825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781864224577058-825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781864224577058-825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 10:17:40.535941 8948 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 10:17:40.536132 8948 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 10:17:40.536154 8948 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 10:17:40.537269 8948 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 10:17:40.537312 8948 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 10:17:40.537325 8948 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 10:17:40.537338 8948 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 10:17:40.537395 8948 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 10:17:40.538875 8948 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.206.84:38663
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781864224577058-825-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781864224577058-825-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.206.126:45539
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781864224577058-825-0/krb5kdc/kudu_127.0.206.126.keytab
--principal=kudu/127.0.206.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.206.126:45539
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781864224577058-825-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.206.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8948
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781864224577058-825-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 09:32:44 UTC on ccycloud.quasar-gbtldp.root.comops.site
build id 4455440
I20260619 10:17:40.539120 8948 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 10:17:40.539348 8948 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260619 10:17:40.544270 8948 server_base.cc:1034] running on GCE node
W20260619 10:17:40.544243 8954 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 10:17:40.544229 8953 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 10:17:40.544240 8956 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 10:17:40.544727 8948 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 10:17:40.545017 8948 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 10:17:40.546188 8948 hybrid_clock.cc:648] HybridClock initialized: now 1781864260546185 us; error 44 us; skew 500 ppm
Jun 19 10:17:40 dist-test-slave-6pt2 krb5kdc[8935](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864260, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 10:17:40.551430 8948 init.cc:377] Logged in from keytab as kudu/127.0.206.126@KRBTEST.COM (short username kudu)
I20260619 10:17:40.551833 8948 webserver.cc:460] Webserver started at http://127.0.206.126:34835/ using document root <none> and password file <none>
I20260619 10:17:40.552059 8948 fs_manager.cc:359] Metadata directory not provided
I20260619 10:17:40.552124 8948 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 10:17:40.552271 8948 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 10:17:40.553305 8948 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781864224577058-825-0/minicluster-data/master-0/data/instance:
uuid: "c241b1e38acc4e30bbb0cfe52c9f3cd7"
format_stamp: "Formatted at 2026-06-19 10:17:40 on dist-test-slave-6pt2"
server_key: "8b4c8658231d151f7ae85b174e2a995d"
server_key_iv: "ec35f00394354823be1cb8acb767eae6"
server_key_version: "encryptionkey@0"
I20260619 10:17:40.553666 8948 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781864224577058-825-0/minicluster-data/master-0/wal/instance:
uuid: "c241b1e38acc4e30bbb0cfe52c9f3cd7"
format_stamp: "Formatted at 2026-06-19 10:17:40 on dist-test-slave-6pt2"
server_key: "8b4c8658231d151f7ae85b174e2a995d"
server_key_iv: "ec35f00394354823be1cb8acb767eae6"
server_key_version: "encryptionkey@0"
I20260619 10:17:40.555012 8948 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 10:17:40.555701 8963 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 10:17:40.555853 8948 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 10:17:40.555968 8948 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781864224577058-825-0/minicluster-data/master-0/data,/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781864224577058-825-0/minicluster-data/master-0/wal
uuid: "c241b1e38acc4e30bbb0cfe52c9f3cd7"
format_stamp: "Formatted at 2026-06-19 10:17:40 on dist-test-slave-6pt2"
server_key: "8b4c8658231d151f7ae85b174e2a995d"
server_key_iv: "ec35f00394354823be1cb8acb767eae6"
server_key_version: "encryptionkey@0"
I20260619 10:17:40.556043 8948 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781864224577058-825-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781864224577058-825-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781864224577058-825-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 10:17:40.579195 8948 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 10:17:40.581535 8948 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 10:17:40.581691 8948 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 10:17:40.585479 8948 rpc_server.cc:307] RPC server started. Bound to: 127.0.206.126:45539
I20260619 10:17:40.585513 9015 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.206.126:45539 every 8 connection(s)
I20260619 10:17:40.585999 8948 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781864224577058-825-0/minicluster-data/master-0/data/info.pb
I20260619 10:17:40.586591 9016 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 10:17:40.588769 9016 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c241b1e38acc4e30bbb0cfe52c9f3cd7: Bootstrap starting.
I20260619 10:17:40.589366 9016 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c241b1e38acc4e30bbb0cfe52c9f3cd7: Neither blocks nor log segments found. Creating new log.
I20260619 10:17:40.589653 9016 log.cc:824] T 00000000000000000000000000000000 P c241b1e38acc4e30bbb0cfe52c9f3cd7: Log is configured to *not* fsync() on all Append() calls
I20260619 10:17:40.590289 9016 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c241b1e38acc4e30bbb0cfe52c9f3cd7: No bootstrap required, opened a new log
I20260619 10:17:40.591863 9016 raft_consensus.cc:348] T 00000000000000000000000000000000 P c241b1e38acc4e30bbb0cfe52c9f3cd7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c241b1e38acc4e30bbb0cfe52c9f3cd7" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 45539 } }
I20260619 10:17:40.592005 9016 raft_consensus.cc:374] T 00000000000000000000000000000000 P c241b1e38acc4e30bbb0cfe52c9f3cd7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 10:17:40.592027 9016 raft_consensus.cc:729] T 00000000000000000000000000000000 P c241b1e38acc4e30bbb0cfe52c9f3cd7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c241b1e38acc4e30bbb0cfe52c9f3cd7, State: Initialized, Role: FOLLOWER
I20260619 10:17:40.592190 9016 consensus_queue.cc:260] T 00000000000000000000000000000000 P c241b1e38acc4e30bbb0cfe52c9f3cd7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c241b1e38acc4e30bbb0cfe52c9f3cd7" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 45539 } }
I20260619 10:17:40.592267 9016 raft_consensus.cc:388] T 00000000000000000000000000000000 P c241b1e38acc4e30bbb0cfe52c9f3cd7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 10:17:40.592321 9016 raft_consensus.cc:482] T 00000000000000000000000000000000 P c241b1e38acc4e30bbb0cfe52c9f3cd7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 10:17:40.592368 9016 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c241b1e38acc4e30bbb0cfe52c9f3cd7 [term 0 FOLLOWER]: Advancing to term 1
I20260619 10:17:40.593178 9016 raft_consensus.cc:504] T 00000000000000000000000000000000 P c241b1e38acc4e30bbb0cfe52c9f3cd7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c241b1e38acc4e30bbb0cfe52c9f3cd7" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 45539 } }
I20260619 10:17:40.593281 9016 leader_election.cc:302] T 00000000000000000000000000000000 P c241b1e38acc4e30bbb0cfe52c9f3cd7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c241b1e38acc4e30bbb0cfe52c9f3cd7; no voters:
I20260619 10:17:40.593456 9016 leader_election.cc:288] T 00000000000000000000000000000000 P c241b1e38acc4e30bbb0cfe52c9f3cd7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 10:17:40.593701 9016 sys_catalog.cc:564] T 00000000000000000000000000000000 P c241b1e38acc4e30bbb0cfe52c9f3cd7 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 10:17:40.593695 825 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTU9OP6/build/release/bin/kudu as pid 8948
I20260619 10:17:40.593808 825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781864224577058-825-0/minicluster-data/master-0/wal/instance
I20260619 10:17:40.593984 825 external_mini_cluster.cc:1428] Setting key a166ac7209373f3550c2713d6400b377
I20260619 10:17:40.594103 9019 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c241b1e38acc4e30bbb0cfe52c9f3cd7 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 10:17:40.594682 9019 raft_consensus.cc:686] T 00000000000000000000000000000000 P c241b1e38acc4e30bbb0cfe52c9f3cd7 [term 1 LEADER]: Becoming Leader. State: Replica: c241b1e38acc4e30bbb0cfe52c9f3cd7, State: Running, Role: LEADER
I20260619 10:17:40.594796 9019 consensus_queue.cc:237] T 00000000000000000000000000000000 P c241b1e38acc4e30bbb0cfe52c9f3cd7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c241b1e38acc4e30bbb0cfe52c9f3cd7" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 45539 } }
I20260619 10:17:40.595302 9020 sys_catalog.cc:455] T 00000000000000000000000000000000 P c241b1e38acc4e30bbb0cfe52c9f3cd7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c241b1e38acc4e30bbb0cfe52c9f3cd7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c241b1e38acc4e30bbb0cfe52c9f3cd7" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 45539 } } }
I20260619 10:17:40.595391 9020 sys_catalog.cc:458] T 00000000000000000000000000000000 P c241b1e38acc4e30bbb0cfe52c9f3cd7 [sys.catalog]: This master's current role is: LEADER
W20260619 10:17:40.595952 9035 catalog_manager.cc:1558] T 00000000000000000000000000000000 P c241b1e38acc4e30bbb0cfe52c9f3cd7: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 10:17:40.596009 9035 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 10:17:40.595610 9024 sys_catalog.cc:455] T 00000000000000000000000000000000 P c241b1e38acc4e30bbb0cfe52c9f3cd7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c241b1e38acc4e30bbb0cfe52c9f3cd7. Latest consensus state: current_term: 1 leader_uuid: "c241b1e38acc4e30bbb0cfe52c9f3cd7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c241b1e38acc4e30bbb0cfe52c9f3cd7" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 45539 } } }
I20260619 10:17:40.596051 9036 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 10:17:40.596217 9036 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 10:17:40.596055 9024 sys_catalog.cc:458] T 00000000000000000000000000000000 P c241b1e38acc4e30bbb0cfe52c9f3cd7 [sys.catalog]: This master's current role is: LEADER
I20260619 10:17:40.597704 9036 catalog_manager.cc:1347] Generated new cluster ID: f1ab41b4885048bd83d26fc6a3da9339
I20260619 10:17:40.597757 9036 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Jun 19 10:17:40 dist-test-slave-6pt2 krb5kdc[8935](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864260, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.206.126@KRBTEST.COM
I20260619 10:17:40.605481 9028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:40.594719 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35418 (local address 127.0.206.126:45539)
0619 10:17:40.594942 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0619 10:17:40.594952 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0619 10:17:40.594982 (+ 30us) server_negotiation.cc:407] Connection header received
0619 10:17:40.595101 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:17:40.595109 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:17:40.595171 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:17:40.596135 (+ 964us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:17:40.596230 (+ 95us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:40.597584 (+ 1354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:40.598670 (+ 1086us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:40.599284 (+ 614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:40.599634 (+ 350us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:40.601886 (+ 2252us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 10:17:40.601889 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 10:17:40.601890 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 10:17:40.601900 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0619 10:17:40.603758 (+ 1858us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:17:40.604120 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:17:40.604123 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:17:40.604123 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:17:40.604210 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:17:40.604377 (+ 167us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:17:40.604377 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:17:40.604377 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:17:40.604635 (+ 258us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 10:17:40.605092 (+ 457us) server_negotiation.cc:1035] Waiting for connection context
0619 10:17:40.605157 (+ 65us) server_negotiation.cc:299] Negotiation successful
0619 10:17:40.605273 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":180,"thread_start_us":95,"threads_started":1}
I20260619 10:17:40.609704 9036 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 10:17:40.610414 9036 catalog_manager.cc:1504] Loading token signing keys...
I20260619 10:17:40.618817 9036 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c241b1e38acc4e30bbb0cfe52c9f3cd7: Generated new TSK 0
I20260619 10:17:40.618980 9036 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.0.206.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.206.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.206.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781864224577058-825-0/krb5kdc/kudu_127.0.206.65.keytab.
Entry for principal kudu/127.0.206.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781864224577058-825-0/krb5kdc/kudu_127.0.206.65.keytab.
I20260619 10:17:40.643394 825 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTU9OP6/build/release/bin/kudu
/tmp/dist-test-taskTU9OP6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781864224577058-825-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781864224577058-825-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781864224577058-825-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781864224577058-825-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.206.65:0
--local_ip_for_outbound_sockets=127.0.206.65
--webserver_interface=127.0.206.65
--webserver_port=0
--tserver_master_addrs=127.0.206.126:45539
--keytab_file=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781864224577058-825-0/krb5kdc/kudu_127.0.206.65.keytab
--principal=kudu/127.0.206.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.206.84:38663
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781864224577058-825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781864224577058-825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781864224577058-825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 10:17:40.757090 9042 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 10:17:40.757349 9042 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 10:17:40.757383 9042 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 10:17:40.759090 9042 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 10:17:40.759145 9042 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 10:17:40.759184 9042 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.206.65
I20260619 10:17:40.761564 9042 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.206.84:38663
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781864224577058-825-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781864224577058-825-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781864224577058-825-0/krb5kdc/kudu_127.0.206.65.keytab
--principal=kudu/127.0.206.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.206.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781864224577058-825-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.206.65
--webserver_port=0
--tserver_master_addrs=127.0.206.126:45539
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9042
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.206.65
--log_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781864224577058-825-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 09:32:44 UTC on ccycloud.quasar-gbtldp.root.comops.site
build id 4455440
I20260619 10:17:40.761794 9042 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 10:17:40.762018 9042 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 10:17:40.766942 9050 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 10:17:40.766964 9048 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 10:17:40.766964 9047 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 10:17:40.767417 9042 server_base.cc:1034] running on GCE node
I20260619 10:17:40.767647 9042 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 10:17:40.767879 9042 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 10:17:40.769080 9042 hybrid_clock.cc:648] HybridClock initialized: now 1781864260769048 us; error 48 us; skew 500 ppm
Jun 19 10:17:40 dist-test-slave-6pt2 krb5kdc[8935](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864260, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 10:17:40.771481 9042 init.cc:377] Logged in from keytab as kudu/127.0.206.65@KRBTEST.COM (short username kudu)
I20260619 10:17:40.771863 9042 webserver.cc:460] Webserver started at http://127.0.206.65:46105/ using document root <none> and password file <none>
I20260619 10:17:40.772034 9042 fs_manager.cc:359] Metadata directory not provided
I20260619 10:17:40.772071 9042 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 10:17:40.772161 9042 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 10:17:40.773209 9042 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781864224577058-825-0/minicluster-data/ts-0/data/instance:
uuid: "83f0ef19785c4a7994421553234e5453"
format_stamp: "Formatted at 2026-06-19 10:17:40 on dist-test-slave-6pt2"
server_key: "fed6810f3a9bc56921f8762f0885912e"
server_key_iv: "6d66e248534878e8a91a757c6aa8f921"
server_key_version: "encryptionkey@0"
I20260619 10:17:40.773499 9042 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781864224577058-825-0/minicluster-data/ts-0/wal/instance:
uuid: "83f0ef19785c4a7994421553234e5453"
format_stamp: "Formatted at 2026-06-19 10:17:40 on dist-test-slave-6pt2"
server_key: "fed6810f3a9bc56921f8762f0885912e"
server_key_iv: "6d66e248534878e8a91a757c6aa8f921"
server_key_version: "encryptionkey@0"
I20260619 10:17:40.774760 9042 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260619 10:17:40.775446 9057 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 10:17:40.775600 9042 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 10:17:40.775679 9042 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781864224577058-825-0/minicluster-data/ts-0/data,/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781864224577058-825-0/minicluster-data/ts-0/wal
uuid: "83f0ef19785c4a7994421553234e5453"
format_stamp: "Formatted at 2026-06-19 10:17:40 on dist-test-slave-6pt2"
server_key: "fed6810f3a9bc56921f8762f0885912e"
server_key_iv: "6d66e248534878e8a91a757c6aa8f921"
server_key_version: "encryptionkey@0"
I20260619 10:17:40.775738 9042 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781864224577058-825-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781864224577058-825-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781864224577058-825-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 10:17:40.784243 9042 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 10:17:40.785817 9042 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 10:17:40.785910 9042 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 10:17:40.786084 9042 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 10:17:40.786293 9042 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 10:17:40.786319 9042 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 10:17:40.786338 9042 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 10:17:40.786350 9042 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 10:17:40.791712 9042 rpc_server.cc:307] RPC server started. Bound to: 127.0.206.65:37695
I20260619 10:17:40.791781 9170 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.206.65:37695 every 8 connection(s)
I20260619 10:17:40.792083 9042 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781864224577058-825-0/minicluster-data/ts-0/data/info.pb
Jun 19 10:17:40 dist-test-slave-6pt2 krb5kdc[8935](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864260, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.65@KRBTEST.COM for kudu/127.0.206.126@KRBTEST.COM
I20260619 10:17:40.798638 825 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTU9OP6/build/release/bin/kudu as pid 9042
I20260619 10:17:40.798784 825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781864224577058-825-0/minicluster-data/ts-0/wal/instance
I20260619 10:17:40.798991 825 external_mini_cluster.cc:1428] Setting key d4fcab2510b1ef430bd25c0522afbb04
I20260619 10:17:40.801277 9028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:40.792762 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.65:51301 (local address 127.0.206.126:45539)
0619 10:17:40.792820 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0619 10:17:40.792822 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 10:17:40.793017 (+ 195us) server_negotiation.cc:407] Connection header received
0619 10:17:40.793999 (+ 982us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:17:40.794000 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:17:40.794015 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:17:40.794064 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:17:40.794738 (+ 674us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:40.795220 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:40.796178 (+ 958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:40.796373 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:40.796443 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:40.798304 (+ 1861us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 10:17:40.798305 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 10:17:40.798305 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 10:17:40.798313 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 10:17:40.800010 (+ 1697us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:17:40.800371 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:17:40.800373 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:17:40.800373 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:17:40.800430 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:17:40.800636 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:17:40.800637 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:17:40.800637 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:17:40.800777 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 10:17:40.800841 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0619 10:17:40.801110 (+ 269us) server_negotiation.cc:299] Negotiation successful
0619 10:17:40.801187 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260619 10:17:40.801461 9173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:40.792747 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:45539 (local address 127.0.206.65:51301)
0619 10:17:40.792911 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0619 10:17:40.792941 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0619 10:17:40.793890 (+ 949us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 10:17:40.794143 (+ 253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 10:17:40.794146 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 10:17:40.794383 (+ 237us) client_negotiation.cc:189] Negotiated authn=SASL
0619 10:17:40.794637 (+ 254us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:40.794642 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:40.795317 (+ 675us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:40.795318 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:40.796081 (+ 763us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:40.796083 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:40.796458 (+ 375us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:40.796459 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:40.796573 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:40.796958 (+ 385us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 10:17:40.796965 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0619 10:17:40.798226 (+ 1261us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 10:17:40.800086 (+ 1860us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 10:17:40.800092 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 10:17:40.800093 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 10:17:40.800294 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 10:17:40.800491 (+ 197us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 10:17:40.800492 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 10:17:40.800492 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 10:17:40.800577 (+ 85us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 10:17:40.800853 (+ 276us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 10:17:40.800858 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 10:17:40.800974 (+ 116us) client_negotiation.cc:769] Sending connection context
0619 10:17:40.801073 (+ 99us) client_negotiation.cc:240] Negotiation successful
0619 10:17:40.801246 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":83,"threads_started":1}
I20260619 10:17:40.801914 9171 heartbeater.cc:347] Connected to a master server at 127.0.206.126:45539
I20260619 10:17:40.801987 9171 heartbeater.cc:464] Registering TS with master...
I20260619 10:17:40.802168 9171 heartbeater.cc:511] Master 127.0.206.126:45539 requested a full tablet report, sending...
I20260619 10:17:40.802690 8980 ts_manager.cc:194] Registered new tserver with Master: 83f0ef19785c4a7994421553234e5453 (127.0.206.65:37695)
I20260619 10:17:40.803293 8980 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.206.65@KRBTEST.COM'} at 127.0.206.65:51301
WARNING: no policy specified for kudu/127.0.206.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.206.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.206.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781864224577058-825-0/krb5kdc/kudu_127.0.206.66.keytab.
Entry for principal kudu/127.0.206.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781864224577058-825-0/krb5kdc/kudu_127.0.206.66.keytab.
I20260619 10:17:40.823036 825 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTU9OP6/build/release/bin/kudu
/tmp/dist-test-taskTU9OP6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781864224577058-825-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781864224577058-825-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781864224577058-825-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781864224577058-825-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.206.66:0
--local_ip_for_outbound_sockets=127.0.206.66
--webserver_interface=127.0.206.66
--webserver_port=0
--tserver_master_addrs=127.0.206.126:45539
--keytab_file=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781864224577058-825-0/krb5kdc/kudu_127.0.206.66.keytab
--principal=kudu/127.0.206.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.206.84:38663
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781864224577058-825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781864224577058-825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781864224577058-825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 10:17:40.901036 9176 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 10:17:40.901212 9176 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 10:17:40.901233 9176 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 10:17:40.902281 9176 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 10:17:40.902315 9176 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 10:17:40.902340 9176 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.206.66
I20260619 10:17:40.903769 9176 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.206.84:38663
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781864224577058-825-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781864224577058-825-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781864224577058-825-0/krb5kdc/kudu_127.0.206.66.keytab
--principal=kudu/127.0.206.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.206.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781864224577058-825-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.206.66
--webserver_port=0
--tserver_master_addrs=127.0.206.126:45539
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9176
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.206.66
--log_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781864224577058-825-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 09:32:44 UTC on ccycloud.quasar-gbtldp.root.comops.site
build id 4455440
I20260619 10:17:40.903939 9176 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 10:17:40.904090 9176 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 10:17:40.908787 9181 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 10:17:40.908824 9182 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 10:17:40.908787 9184 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 10:17:40.908908 9176 server_base.cc:1034] running on GCE node
I20260619 10:17:40.909106 9176 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 10:17:40.909358 9176 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 10:17:40.910506 9176 hybrid_clock.cc:648] HybridClock initialized: now 1781864260910503 us; error 35 us; skew 500 ppm
Jun 19 10:17:40 dist-test-slave-6pt2 krb5kdc[8935](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864260, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 10:17:40.912799 9176 init.cc:377] Logged in from keytab as kudu/127.0.206.66@KRBTEST.COM (short username kudu)
I20260619 10:17:40.913233 9176 webserver.cc:460] Webserver started at http://127.0.206.66:33079/ using document root <none> and password file <none>
I20260619 10:17:40.913439 9176 fs_manager.cc:359] Metadata directory not provided
I20260619 10:17:40.913502 9176 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 10:17:40.913648 9176 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 10:17:40.914673 9176 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781864224577058-825-0/minicluster-data/ts-1/data/instance:
uuid: "5c2d08d0ab8e4118885f8b5209af023b"
format_stamp: "Formatted at 2026-06-19 10:17:40 on dist-test-slave-6pt2"
server_key: "b57e11c2f41bce8c18d1d2dd6bb0b4e8"
server_key_iv: "3e5bcb2c8062c1afca745a74693f6acc"
server_key_version: "encryptionkey@0"
I20260619 10:17:40.915028 9176 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781864224577058-825-0/minicluster-data/ts-1/wal/instance:
uuid: "5c2d08d0ab8e4118885f8b5209af023b"
format_stamp: "Formatted at 2026-06-19 10:17:40 on dist-test-slave-6pt2"
server_key: "b57e11c2f41bce8c18d1d2dd6bb0b4e8"
server_key_iv: "3e5bcb2c8062c1afca745a74693f6acc"
server_key_version: "encryptionkey@0"
I20260619 10:17:40.916216 9176 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260619 10:17:40.916826 9191 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 10:17:40.917021 9176 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260619 10:17:40.917106 9176 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781864224577058-825-0/minicluster-data/ts-1/data,/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781864224577058-825-0/minicluster-data/ts-1/wal
uuid: "5c2d08d0ab8e4118885f8b5209af023b"
format_stamp: "Formatted at 2026-06-19 10:17:40 on dist-test-slave-6pt2"
server_key: "b57e11c2f41bce8c18d1d2dd6bb0b4e8"
server_key_iv: "3e5bcb2c8062c1afca745a74693f6acc"
server_key_version: "encryptionkey@0"
I20260619 10:17:40.917158 9176 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781864224577058-825-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781864224577058-825-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781864224577058-825-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 10:17:40.926525 9176 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 10:17:40.928686 9176 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 10:17:40.928786 9176 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 10:17:40.928943 9176 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 10:17:40.929157 9176 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 10:17:40.929185 9176 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 10:17:40.929205 9176 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 10:17:40.929219 9176 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 10:17:40.935483 9176 rpc_server.cc:307] RPC server started. Bound to: 127.0.206.66:38513
I20260619 10:17:40.935794 9176 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781864224577058-825-0/minicluster-data/ts-1/data/info.pb
I20260619 10:17:40.936240 9304 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.206.66:38513 every 8 connection(s)
I20260619 10:17:40.937207 825 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTU9OP6/build/release/bin/kudu as pid 9176
I20260619 10:17:40.937309 825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781864224577058-825-0/minicluster-data/ts-1/wal/instance
I20260619 10:17:40.937516 825 external_mini_cluster.cc:1428] Setting key 9f543be8de31e4a632fbf8f7419a9ec2
Jun 19 10:17:40 dist-test-slave-6pt2 krb5kdc[8935](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864260, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.66@KRBTEST.COM for kudu/127.0.206.126@KRBTEST.COM
I20260619 10:17:40.946815 9028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:40.936895 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.66:38183 (local address 127.0.206.126:45539)
0619 10:17:40.936953 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0619 10:17:40.936955 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 10:17:40.936967 (+ 12us) server_negotiation.cc:407] Connection header received
0619 10:17:40.938180 (+ 1213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:17:40.938182 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:17:40.938196 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:17:40.938245 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:17:40.938962 (+ 717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:40.939662 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:40.940698 (+ 1036us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:40.940935 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:40.941015 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:40.944040 (+ 3025us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 10:17:40.944042 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 10:17:40.944042 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 10:17:40.944049 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 10:17:40.945454 (+ 1405us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:17:40.945862 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:17:40.945863 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:17:40.945864 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:17:40.945921 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:17:40.946173 (+ 252us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:17:40.946174 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:17:40.946175 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:17:40.946292 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 10:17:40.946367 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0619 10:17:40.946644 (+ 277us) server_negotiation.cc:299] Negotiation successful
0619 10:17:40.946722 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
WARNING: no policy specified for kudu/127.0.206.67@KRBTEST.COM; defaulting to no policy
I20260619 10:17:40.947007 9307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:40.936417 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:45539 (local address 127.0.206.66:38183)
0619 10:17:40.936628 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0619 10:17:40.936689 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0619 10:17:40.938057 (+ 1368us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 10:17:40.938326 (+ 269us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 10:17:40.938330 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 10:17:40.938566 (+ 236us) client_negotiation.cc:189] Negotiated authn=SASL
0619 10:17:40.938872 (+ 306us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:40.938877 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:40.939760 (+ 883us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:40.939761 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:40.940598 (+ 837us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:40.940601 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:40.941024 (+ 423us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:40.941025 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:40.941189 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:40.942073 (+ 884us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 10:17:40.942099 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0619 10:17:40.943923 (+ 1824us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 10:17:40.945551 (+ 1628us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 10:17:40.945557 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 10:17:40.945559 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 10:17:40.945776 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 10:17:40.945984 (+ 208us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 10:17:40.945985 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 10:17:40.945985 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 10:17:40.946091 (+ 106us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 10:17:40.946358 (+ 267us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 10:17:40.946364 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 10:17:40.946498 (+ 134us) client_negotiation.cc:769] Sending connection context
0619 10:17:40.946604 (+ 106us) client_negotiation.cc:240] Negotiation successful
0619 10:17:40.946780 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":171,"thread_start_us":96,"threads_started":1}
I20260619 10:17:40.947538 9305 heartbeater.cc:347] Connected to a master server at 127.0.206.126:45539
I20260619 10:17:40.947628 9305 heartbeater.cc:464] Registering TS with master...
I20260619 10:17:40.947832 9305 heartbeater.cc:511] Master 127.0.206.126:45539 requested a full tablet report, sending...
I20260619 10:17:40.948202 8980 ts_manager.cc:194] Registered new tserver with Master: 5c2d08d0ab8e4118885f8b5209af023b (127.0.206.66:38513)
I20260619 10:17:40.948523 8980 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.206.66@KRBTEST.COM'} at 127.0.206.66:38183
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.206.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.206.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781864224577058-825-0/krb5kdc/kudu_127.0.206.67.keytab.
Entry for principal kudu/127.0.206.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781864224577058-825-0/krb5kdc/kudu_127.0.206.67.keytab.
I20260619 10:17:40.961246 825 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTU9OP6/build/release/bin/kudu
/tmp/dist-test-taskTU9OP6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781864224577058-825-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781864224577058-825-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781864224577058-825-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781864224577058-825-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.206.67:0
--local_ip_for_outbound_sockets=127.0.206.67
--webserver_interface=127.0.206.67
--webserver_port=0
--tserver_master_addrs=127.0.206.126:45539
--keytab_file=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781864224577058-825-0/krb5kdc/kudu_127.0.206.67.keytab
--principal=kudu/127.0.206.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.206.84:38663
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781864224577058-825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781864224577058-825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781864224577058-825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 10:17:41.040026 9310 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 10:17:41.040252 9310 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 10:17:41.040290 9310 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 10:17:41.041461 9310 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 10:17:41.041528 9310 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 10:17:41.041572 9310 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.206.67
I20260619 10:17:41.043144 9310 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.206.84:38663
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781864224577058-825-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781864224577058-825-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781864224577058-825-0/krb5kdc/kudu_127.0.206.67.keytab
--principal=kudu/127.0.206.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.206.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781864224577058-825-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.206.67
--webserver_port=0
--tserver_master_addrs=127.0.206.126:45539
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9310
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.206.67
--log_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781864224577058-825-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 09:32:44 UTC on ccycloud.quasar-gbtldp.root.comops.site
build id 4455440
I20260619 10:17:41.043411 9310 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 10:17:41.043635 9310 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 10:17:41.048491 9315 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 10:17:41.048518 9310 server_base.cc:1034] running on GCE node
W20260619 10:17:41.048519 9316 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 10:17:41.048491 9318 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 10:17:41.048906 9310 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 10:17:41.049150 9310 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 10:17:41.050300 9310 hybrid_clock.cc:648] HybridClock initialized: now 1781864261050299 us; error 33 us; skew 500 ppm
Jun 19 10:17:41 dist-test-slave-6pt2 krb5kdc[8935](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864261, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 10:17:41.052762 9310 init.cc:377] Logged in from keytab as kudu/127.0.206.67@KRBTEST.COM (short username kudu)
I20260619 10:17:41.053179 9310 webserver.cc:460] Webserver started at http://127.0.206.67:42637/ using document root <none> and password file <none>
I20260619 10:17:41.053354 9310 fs_manager.cc:359] Metadata directory not provided
I20260619 10:17:41.053395 9310 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 10:17:41.053478 9310 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 10:17:41.054433 9310 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781864224577058-825-0/minicluster-data/ts-2/data/instance:
uuid: "d5aac998d97c49a1bbf6bfe55ba25065"
format_stamp: "Formatted at 2026-06-19 10:17:41 on dist-test-slave-6pt2"
server_key: "8473c3cddefdd8bebbc02ab0285a3d78"
server_key_iv: "0dbe17ddeade59498194bfde36a10171"
server_key_version: "encryptionkey@0"
I20260619 10:17:41.054809 9310 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781864224577058-825-0/minicluster-data/ts-2/wal/instance:
uuid: "d5aac998d97c49a1bbf6bfe55ba25065"
format_stamp: "Formatted at 2026-06-19 10:17:41 on dist-test-slave-6pt2"
server_key: "8473c3cddefdd8bebbc02ab0285a3d78"
server_key_iv: "0dbe17ddeade59498194bfde36a10171"
server_key_version: "encryptionkey@0"
I20260619 10:17:41.056052 9310 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260619 10:17:41.056764 9325 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 10:17:41.056941 9310 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260619 10:17:41.057011 9310 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781864224577058-825-0/minicluster-data/ts-2/data,/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781864224577058-825-0/minicluster-data/ts-2/wal
uuid: "d5aac998d97c49a1bbf6bfe55ba25065"
format_stamp: "Formatted at 2026-06-19 10:17:41 on dist-test-slave-6pt2"
server_key: "8473c3cddefdd8bebbc02ab0285a3d78"
server_key_iv: "0dbe17ddeade59498194bfde36a10171"
server_key_version: "encryptionkey@0"
I20260619 10:17:41.057096 9310 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781864224577058-825-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781864224577058-825-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781864224577058-825-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 10:17:41.065371 9310 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 10:17:41.067557 9310 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 10:17:41.067704 9310 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 10:17:41.067917 9310 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 10:17:41.068186 9310 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 10:17:41.068217 9310 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 10:17:41.068257 9310 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 10:17:41.068284 9310 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 10:17:41.073765 9310 rpc_server.cc:307] RPC server started. Bound to: 127.0.206.67:36771
I20260619 10:17:41.073835 9438 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.206.67:36771 every 8 connection(s)
I20260619 10:17:41.074092 9310 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781864224577058-825-0/minicluster-data/ts-2/data/info.pb
I20260619 10:17:41.075182 825 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTU9OP6/build/release/bin/kudu as pid 9310
I20260619 10:17:41.075305 825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781864224577058-825-0/minicluster-data/ts-2/wal/instance
I20260619 10:17:41.075502 825 external_mini_cluster.cc:1428] Setting key ae59e9e7f4d7f29491ea009a02701752
Jun 19 10:17:41 dist-test-slave-6pt2 krb5kdc[8935](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864261, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.67@KRBTEST.COM for kudu/127.0.206.126@KRBTEST.COM
I20260619 10:17:41.083138 9028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:41.074806 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.67:48919 (local address 127.0.206.126:45539)
0619 10:17:41.074874 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0619 10:17:41.074878 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 10:17:41.075105 (+ 227us) server_negotiation.cc:407] Connection header received
0619 10:17:41.076414 (+ 1309us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:17:41.076415 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:17:41.076428 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:17:41.076473 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:17:41.077000 (+ 527us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:41.077463 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:41.078477 (+ 1014us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:41.078727 (+ 250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:41.078782 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:41.080573 (+ 1791us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 10:17:41.080574 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 10:17:41.080575 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 10:17:41.080581 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 10:17:41.081946 (+ 1365us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:17:41.082278 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:17:41.082279 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:17:41.082279 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:17:41.082324 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:17:41.082508 (+ 184us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:17:41.082508 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:17:41.082509 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:17:41.082732 (+ 223us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 10:17:41.082791 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0619 10:17:41.083013 (+ 222us) server_negotiation.cc:299] Negotiation successful
0619 10:17:41.083076 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260619 10:17:41.083298 9441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:41.074795 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:45539 (local address 127.0.206.67:48919)
0619 10:17:41.074969 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0619 10:17:41.075015 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0619 10:17:41.076324 (+ 1309us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 10:17:41.076539 (+ 215us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 10:17:41.076546 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 10:17:41.076763 (+ 217us) client_negotiation.cc:189] Negotiated authn=SASL
0619 10:17:41.076934 (+ 171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:41.076938 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:41.077533 (+ 595us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:41.077534 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:41.078402 (+ 868us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:41.078405 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:41.078788 (+ 383us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:41.078788 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:41.078890 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:41.079235 (+ 345us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 10:17:41.079242 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0619 10:17:41.080500 (+ 1258us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 10:17:41.082011 (+ 1511us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 10:17:41.082015 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 10:17:41.082016 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 10:17:41.082202 (+ 186us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 10:17:41.082376 (+ 174us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 10:17:41.082377 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 10:17:41.082377 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 10:17:41.082459 (+ 82us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 10:17:41.082794 (+ 335us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 10:17:41.082798 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 10:17:41.082909 (+ 111us) client_negotiation.cc:769] Sending connection context
0619 10:17:41.082986 (+ 77us) client_negotiation.cc:240] Negotiation successful
0619 10:17:41.083092 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":102,"threads_started":1}
I20260619 10:17:41.083725 9439 heartbeater.cc:347] Connected to a master server at 127.0.206.126:45539
I20260619 10:17:41.083819 9439 heartbeater.cc:464] Registering TS with master...
I20260619 10:17:41.083992 9439 heartbeater.cc:511] Master 127.0.206.126:45539 requested a full tablet report, sending...
I20260619 10:17:41.084342 8980 ts_manager.cc:194] Registered new tserver with Master: d5aac998d97c49a1bbf6bfe55ba25065 (127.0.206.67:36771)
I20260619 10:17:41.084688 8980 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.206.67@KRBTEST.COM'} at 127.0.206.67:48919
I20260619 10:17:41.087251 825 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.0.206.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.0.206.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.0.206.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781864224577058-825-0/krb5kdc/oryx_127.0.206.65.keytab.
Entry for principal oryx/127.0.206.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781864224577058-825-0/krb5kdc/oryx_127.0.206.65.keytab.
WARNING: no policy specified for oryx/127.0.206.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.0.206.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.0.206.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781864224577058-825-0/krb5kdc/oryx_127.0.206.66.keytab.
Entry for principal oryx/127.0.206.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781864224577058-825-0/krb5kdc/oryx_127.0.206.66.keytab.
WARNING: no policy specified for oryx/127.0.206.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.0.206.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.0.206.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781864224577058-825-0/krb5kdc/oryx_127.0.206.67.keytab.
Entry for principal oryx/127.0.206.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781864224577058-825-0/krb5kdc/oryx_127.0.206.67.keytab.
I20260619 10:17:41.146463 825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTU9OP6/build/release/bin/kudu with pid 9042
I20260619 10:17:41.151211 825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTU9OP6/build/release/bin/kudu with pid 9176
I20260619 10:17:41.155639 825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTU9OP6/build/release/bin/kudu with pid 9310
I20260619 10:17:41.159628 825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTU9OP6/build/release/bin/kudu with pid 8948
I20260619 10:17:41.164583 825 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTU9OP6/build/release/bin/kudu
/tmp/dist-test-taskTU9OP6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781864224577058-825-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781864224577058-825-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781864224577058-825-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781864224577058-825-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.206.126:45539
--webserver_interface=127.0.206.126
--webserver_port=34835
--keytab_file=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781864224577058-825-0/krb5kdc/kudu_127.0.206.126.keytab
--principal=kudu/127.0.206.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.206.84:38663
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.206.126:45539
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781864224577058-825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781864224577058-825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781864224577058-825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 10:17:41.240516 9448 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 10:17:41.240720 9448 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 10:17:41.240751 9448 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 10:17:41.241856 9448 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 10:17:41.241925 9448 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 10:17:41.241948 9448 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 10:17:41.241969 9448 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 10:17:41.241998 9448 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 10:17:41.243440 9448 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.206.84:38663
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781864224577058-825-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781864224577058-825-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.206.126:45539
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781864224577058-825-0/krb5kdc/kudu_127.0.206.126.keytab
--principal=kudu/127.0.206.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.206.126:45539
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781864224577058-825-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.206.126
--webserver_port=34835
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9448
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781864224577058-825-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 09:32:44 UTC on ccycloud.quasar-gbtldp.root.comops.site
build id 4455440
I20260619 10:17:41.243709 9448 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 10:17:41.243925 9448 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 10:17:41.248469 9456 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 10:17:41.248677 9448 server_base.cc:1034] running on GCE node
W20260619 10:17:41.248452 9454 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 10:17:41.248452 9453 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 10:17:41.249019 9448 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 10:17:41.249307 9448 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 10:17:41.250448 9448 hybrid_clock.cc:648] HybridClock initialized: now 1781864261250438 us; error 32 us; skew 500 ppm
Jun 19 10:17:41 dist-test-slave-6pt2 krb5kdc[8935](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864261, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 10:17:41.252756 9448 init.cc:377] Logged in from keytab as kudu/127.0.206.126@KRBTEST.COM (short username kudu)
I20260619 10:17:41.253170 9448 webserver.cc:460] Webserver started at http://127.0.206.126:34835/ using document root <none> and password file <none>
I20260619 10:17:41.253381 9448 fs_manager.cc:359] Metadata directory not provided
I20260619 10:17:41.253456 9448 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 10:17:41.254720 9448 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260619 10:17:41.255334 9463 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 10:17:41.255501 9448 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260619 10:17:41.255591 9448 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781864224577058-825-0/minicluster-data/master-0/data,/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781864224577058-825-0/minicluster-data/master-0/wal
uuid: "c241b1e38acc4e30bbb0cfe52c9f3cd7"
format_stamp: "Formatted at 2026-06-19 10:17:40 on dist-test-slave-6pt2"
server_key: "8b4c8658231d151f7ae85b174e2a995d"
server_key_iv: "ec35f00394354823be1cb8acb767eae6"
server_key_version: "encryptionkey@0"
I20260619 10:17:41.255892 9448 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781864224577058-825-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781864224577058-825-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781864224577058-825-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 10:17:41.263937 9448 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 10:17:41.265566 9448 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 10:17:41.265714 9448 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 10:17:41.269632 9448 rpc_server.cc:307] RPC server started. Bound to: 127.0.206.126:45539
I20260619 10:17:41.269657 9515 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.206.126:45539 every 8 connection(s)
I20260619 10:17:41.270136 9448 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781864224577058-825-0/minicluster-data/master-0/data/info.pb
I20260619 10:17:41.271413 9516 sys_catalog.cc:263] Verifying existing consensus state
I20260619 10:17:41.271965 9516 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c241b1e38acc4e30bbb0cfe52c9f3cd7: Bootstrap starting.
I20260619 10:17:41.279179 825 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTU9OP6/build/release/bin/kudu as pid 9448
I20260619 10:17:41.279460 825 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTU9OP6/build/release/bin/kudu
/tmp/dist-test-taskTU9OP6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781864224577058-825-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781864224577058-825-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781864224577058-825-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781864224577058-825-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.206.65:37695
--local_ip_for_outbound_sockets=127.0.206.65
--tserver_master_addrs=127.0.206.126:45539
--webserver_port=46105
--webserver_interface=127.0.206.65
--keytab_file=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781864224577058-825-0/krb5kdc/kudu_127.0.206.65.keytab
--principal=kudu/127.0.206.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.206.84:38663
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.0.206.65
--keytab_file=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781864224577058-825-0/krb5kdc/oryx_127.0.206.65.keytab with env {KRB5CCNAME=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781864224577058-825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781864224577058-825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781864224577058-825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 10:17:41.279811 9516 log.cc:824] T 00000000000000000000000000000000 P c241b1e38acc4e30bbb0cfe52c9f3cd7: Log is configured to *not* fsync() on all Append() calls
I20260619 10:17:41.281785 9516 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c241b1e38acc4e30bbb0cfe52c9f3cd7: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260619 10:17:41.282064 9516 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c241b1e38acc4e30bbb0cfe52c9f3cd7: Bootstrap complete.
I20260619 10:17:41.284231 9516 raft_consensus.cc:348] T 00000000000000000000000000000000 P c241b1e38acc4e30bbb0cfe52c9f3cd7 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c241b1e38acc4e30bbb0cfe52c9f3cd7" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 45539 } }
I20260619 10:17:41.284446 9516 raft_consensus.cc:729] T 00000000000000000000000000000000 P c241b1e38acc4e30bbb0cfe52c9f3cd7 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: c241b1e38acc4e30bbb0cfe52c9f3cd7, State: Initialized, Role: FOLLOWER
I20260619 10:17:41.284598 9516 consensus_queue.cc:260] T 00000000000000000000000000000000 P c241b1e38acc4e30bbb0cfe52c9f3cd7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c241b1e38acc4e30bbb0cfe52c9f3cd7" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 45539 } }
I20260619 10:17:41.284668 9516 raft_consensus.cc:388] T 00000000000000000000000000000000 P c241b1e38acc4e30bbb0cfe52c9f3cd7 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 10:17:41.284737 9516 raft_consensus.cc:482] T 00000000000000000000000000000000 P c241b1e38acc4e30bbb0cfe52c9f3cd7 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 10:17:41.284780 9516 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c241b1e38acc4e30bbb0cfe52c9f3cd7 [term 1 FOLLOWER]: Advancing to term 2
I20260619 10:17:41.285697 9516 raft_consensus.cc:504] T 00000000000000000000000000000000 P c241b1e38acc4e30bbb0cfe52c9f3cd7 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c241b1e38acc4e30bbb0cfe52c9f3cd7" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 45539 } }
I20260619 10:17:41.285796 9516 leader_election.cc:302] T 00000000000000000000000000000000 P c241b1e38acc4e30bbb0cfe52c9f3cd7 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c241b1e38acc4e30bbb0cfe52c9f3cd7; no voters:
I20260619 10:17:41.285974 9516 leader_election.cc:288] T 00000000000000000000000000000000 P c241b1e38acc4e30bbb0cfe52c9f3cd7 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260619 10:17:41.286010 9520 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c241b1e38acc4e30bbb0cfe52c9f3cd7 [term 2 FOLLOWER]: Leader election won for term 2
I20260619 10:17:41.286121 9520 raft_consensus.cc:686] T 00000000000000000000000000000000 P c241b1e38acc4e30bbb0cfe52c9f3cd7 [term 2 LEADER]: Becoming Leader. State: Replica: c241b1e38acc4e30bbb0cfe52c9f3cd7, State: Running, Role: LEADER
I20260619 10:17:41.286207 9520 consensus_queue.cc:237] T 00000000000000000000000000000000 P c241b1e38acc4e30bbb0cfe52c9f3cd7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c241b1e38acc4e30bbb0cfe52c9f3cd7" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 45539 } }
I20260619 10:17:41.286211 9516 sys_catalog.cc:564] T 00000000000000000000000000000000 P c241b1e38acc4e30bbb0cfe52c9f3cd7 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 10:17:41.286458 9522 sys_catalog.cc:455] T 00000000000000000000000000000000 P c241b1e38acc4e30bbb0cfe52c9f3cd7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c241b1e38acc4e30bbb0cfe52c9f3cd7. Latest consensus state: current_term: 2 leader_uuid: "c241b1e38acc4e30bbb0cfe52c9f3cd7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c241b1e38acc4e30bbb0cfe52c9f3cd7" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 45539 } } }
I20260619 10:17:41.286521 9522 sys_catalog.cc:458] T 00000000000000000000000000000000 P c241b1e38acc4e30bbb0cfe52c9f3cd7 [sys.catalog]: This master's current role is: LEADER
I20260619 10:17:41.286659 9521 sys_catalog.cc:455] T 00000000000000000000000000000000 P c241b1e38acc4e30bbb0cfe52c9f3cd7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "c241b1e38acc4e30bbb0cfe52c9f3cd7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c241b1e38acc4e30bbb0cfe52c9f3cd7" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 45539 } } }
I20260619 10:17:41.286715 9521 sys_catalog.cc:458] T 00000000000000000000000000000000 P c241b1e38acc4e30bbb0cfe52c9f3cd7 [sys.catalog]: This master's current role is: LEADER
I20260619 10:17:41.286808 9524 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 10:17:41.287263 9524 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 10:17:41.287389 9524 catalog_manager.cc:1259] Loaded cluster ID: f1ab41b4885048bd83d26fc6a3da9339
I20260619 10:17:41.287425 9524 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 10:17:41.288553 9524 catalog_manager.cc:1504] Loading token signing keys...
I20260619 10:17:41.288715 9524 catalog_manager.cc:5879] T 00000000000000000000000000000000 P c241b1e38acc4e30bbb0cfe52c9f3cd7: Loaded TSK: 0
I20260619 10:17:41.288923 9524 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260619 10:17:41.357801 9519 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 10:17:41.357995 9519 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 10:17:41.358031 9519 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 10:17:41.359306 9519 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 10:17:41.359344 9519 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 10:17:41.359370 9519 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.206.65
I20260619 10:17:41.360728 9519 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.206.84:38663
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781864224577058-825-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781864224577058-825-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781864224577058-825-0/krb5kdc/oryx_127.0.206.65.keytab
--principal=oryx/127.0.206.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.206.65:37695
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781864224577058-825-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.206.65
--webserver_port=46105
--tserver_master_addrs=127.0.206.126:45539
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9519
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.206.65
--log_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781864224577058-825-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 09:32:44 UTC on ccycloud.quasar-gbtldp.root.comops.site
build id 4455440
I20260619 10:17:41.360934 9519 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 10:17:41.361168 9519 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 10:17:41.365989 9543 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 10:17:41.366154 9519 server_base.cc:1034] running on GCE node
W20260619 10:17:41.366191 9545 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 10:17:41.365981 9542 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 10:17:41.366493 9519 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 10:17:41.366777 9519 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 10:17:41.367923 9519 hybrid_clock.cc:648] HybridClock initialized: now 1781864261367910 us; error 31 us; skew 500 ppm
Jun 19 10:17:41 dist-test-slave-6pt2 krb5kdc[8935](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864261, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.206.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 10:17:41.370280 9519 init.cc:377] Logged in from keytab as oryx/127.0.206.65@KRBTEST.COM (short username oryx)
I20260619 10:17:41.370743 9519 webserver.cc:460] Webserver started at http://127.0.206.65:46105/ using document root <none> and password file <none>
I20260619 10:17:41.370954 9519 fs_manager.cc:359] Metadata directory not provided
I20260619 10:17:41.371025 9519 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 10:17:41.372293 9519 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260619 10:17:41.372980 9552 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 10:17:41.373140 9519 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260619 10:17:41.373230 9519 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781864224577058-825-0/minicluster-data/ts-0/data,/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781864224577058-825-0/minicluster-data/ts-0/wal
uuid: "83f0ef19785c4a7994421553234e5453"
format_stamp: "Formatted at 2026-06-19 10:17:40 on dist-test-slave-6pt2"
server_key: "fed6810f3a9bc56921f8762f0885912e"
server_key_iv: "6d66e248534878e8a91a757c6aa8f921"
server_key_version: "encryptionkey@0"
I20260619 10:17:41.373543 9519 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781864224577058-825-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781864224577058-825-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781864224577058-825-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 10:17:41.386708 9519 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 10:17:41.388885 9519 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 10:17:41.389034 9519 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 10:17:41.389274 9519 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 10:17:41.389562 9519 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 10:17:41.389616 9519 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 10:17:41.389654 9519 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 10:17:41.389680 9519 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 10:17:41.395854 9519 rpc_server.cc:307] RPC server started. Bound to: 127.0.206.65:37695
I20260619 10:17:41.395896 9665 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.206.65:37695 every 8 connection(s)
I20260619 10:17:41.396220 9519 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781864224577058-825-0/minicluster-data/ts-0/data/info.pb
Jun 19 10:17:41 dist-test-slave-6pt2 krb5kdc[8935](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.206.65@KRBTEST.COM for oryx/127.0.206.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 10:17:41 dist-test-slave-6pt2 krb5kdc[8935](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.206.65@KRBTEST.COM for oryx/127.0.206.126@KRBTEST.COM, Server not found in Kerberos database
I20260619 10:17:41.404458 825 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTU9OP6/build/release/bin/kudu as pid 9519
I20260619 10:17:41.404795 825 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTU9OP6/build/release/bin/kudu
/tmp/dist-test-taskTU9OP6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781864224577058-825-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781864224577058-825-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781864224577058-825-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781864224577058-825-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.206.66:38513
--local_ip_for_outbound_sockets=127.0.206.66
--tserver_master_addrs=127.0.206.126:45539
--webserver_port=33079
--webserver_interface=127.0.206.66
--keytab_file=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781864224577058-825-0/krb5kdc/kudu_127.0.206.66.keytab
--principal=kudu/127.0.206.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.206.84:38663
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.0.206.66
--keytab_file=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781864224577058-825-0/krb5kdc/oryx_127.0.206.66.keytab with env {KRB5CCNAME=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781864224577058-825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781864224577058-825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781864224577058-825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 10:17:41.405277 9668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:41.396972 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:45539 (local address 127.0.206.65:37843)
0619 10:17:41.397143 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0619 10:17:41.397190 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0619 10:17:41.397657 (+ 467us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 10:17:41.398103 (+ 446us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 10:17:41.398106 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 10:17:41.398714 (+ 608us) client_negotiation.cc:189] Negotiated authn=SASL
0619 10:17:41.398905 (+ 191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:41.398910 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:41.400545 (+ 1635us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:41.400545 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:41.401282 (+ 737us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:41.401283 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:41.402022 (+ 739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:41.402022 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:41.402111 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:41.402593 (+ 482us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 10:17:41.402658 (+ 65us) client_negotiation.cc:656] calling sasl_client_start()
0619 10:17:41.405045 (+ 2387us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.206.126:45539: Server oryx/127.0.206.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":76,"threads_started":1}
W20260619 10:17:41.405524 9666 heartbeater.cc:644] Failed to heartbeat to 127.0.206.126:45539 (0 consecutive failures): Not authorized: Failed to ping master at 127.0.206.126:45539: Client connection negotiation failed: client connection to 127.0.206.126:45539: Server oryx/127.0.206.126@KRBTEST.COM not found in Kerberos database
W20260619 10:17:41.405532 9669 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 10:17:41.397051 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.65:37843 (local address 127.0.206.126:45539)
0619 10:17:41.397253 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0619 10:17:41.397262 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 10:17:41.397305 (+ 43us) server_negotiation.cc:407] Connection header received
0619 10:17:41.397864 (+ 559us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:17:41.397871 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:17:41.397926 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:17:41.398063 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:17:41.399022 (+ 959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:41.400453 (+ 1431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:41.401374 (+ 921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:41.401935 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:41.402042 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:41.405188 (+ 3146us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.206.65:37843: BlockingRecv error: recv got EOF from 127.0.206.65:37843 (error 108)
Metrics: {"server-negotiator.queue_time_us":154,"thread_start_us":103,"threads_started":1}
Jun 19 10:17:41 dist-test-slave-6pt2 krb5kdc[8935](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.206.65@KRBTEST.COM for oryx/127.0.206.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 10:17:41 dist-test-slave-6pt2 krb5kdc[8935](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.206.65@KRBTEST.COM for oryx/127.0.206.126@KRBTEST.COM, Server not found in Kerberos database
I20260619 10:17:41.410933 9668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:41.405765 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:45539 (local address 127.0.206.65:40083)
0619 10:17:41.405806 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0619 10:17:41.405815 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0619 10:17:41.405847 (+ 32us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 10:17:41.406054 (+ 207us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 10:17:41.406055 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 10:17:41.406211 (+ 156us) client_negotiation.cc:189] Negotiated authn=SASL
0619 10:17:41.406264 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:41.406266 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:41.407161 (+ 895us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:41.407162 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:41.407982 (+ 820us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:41.407985 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:41.408394 (+ 409us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:41.408395 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:41.408479 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:41.408894 (+ 415us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 10:17:41.408901 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0619 10:17:41.410858 (+ 1957us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.206.126:45539: Server oryx/127.0.206.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":27}
W20260619 10:17:41.410981 9669 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 10:17:41.405824 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.65:40083 (local address 127.0.206.126:45539)
0619 10:17:41.405939 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0619 10:17:41.405941 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 10:17:41.405948 (+ 7us) server_negotiation.cc:407] Connection header received
0619 10:17:41.405966 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:17:41.405967 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:17:41.405978 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:17:41.406027 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:17:41.406341 (+ 314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:41.407063 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:41.408075 (+ 1012us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:41.408313 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:41.408381 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:41.410920 (+ 2539us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.206.65:40083: BlockingRecv error: recv got EOF from 127.0.206.65:40083 (error 108)
Metrics: {"server-negotiator.queue_time_us":102,"thread_start_us":93,"threads_started":1}
Jun 19 10:17:41 dist-test-slave-6pt2 krb5kdc[8935](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.206.65@KRBTEST.COM for oryx/127.0.206.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 10:17:41 dist-test-slave-6pt2 krb5kdc[8935](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.206.65@KRBTEST.COM for oryx/127.0.206.126@KRBTEST.COM, Server not found in Kerberos database
I20260619 10:17:41.415285 9668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:41.411196 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:45539 (local address 127.0.206.65:60855)
0619 10:17:41.411240 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0619 10:17:41.411251 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 10:17:41.411287 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 10:17:41.411432 (+ 145us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 10:17:41.411432 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 10:17:41.411553 (+ 121us) client_negotiation.cc:189] Negotiated authn=SASL
0619 10:17:41.411575 (+ 22us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:41.411576 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:41.412160 (+ 584us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:41.412161 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:41.412642 (+ 481us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:41.412643 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:41.412988 (+ 345us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:41.412989 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:41.413040 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:41.413473 (+ 433us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 10:17:41.413478 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 10:17:41.415210 (+ 1732us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.206.126:45539: Server oryx/127.0.206.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":30}
W20260619 10:17:41.415397 9666 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260619 10:17:41.415349 9669 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 10:17:41.411295 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.65:60855 (local address 127.0.206.126:45539)
0619 10:17:41.411328 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0619 10:17:41.411329 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 10:17:41.411336 (+ 7us) server_negotiation.cc:407] Connection header received
0619 10:17:41.411356 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:17:41.411356 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:17:41.411369 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:17:41.411428 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:17:41.411649 (+ 221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:41.412084 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:41.412730 (+ 646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:41.412908 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:41.412988 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:41.415279 (+ 2291us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.206.65:60855: BlockingRecv error: recv got EOF from 127.0.206.65:60855 (error 108)
Metrics: {"server-negotiator.queue_time_us":16}
W20260619 10:17:41.486562 9671 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 10:17:41.486753 9671 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 10:17:41.486778 9671 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 10:17:41.488006 9671 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 10:17:41.488045 9671 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 10:17:41.488092 9671 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.206.66
I20260619 10:17:41.489427 9671 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.206.84:38663
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781864224577058-825-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781864224577058-825-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781864224577058-825-0/krb5kdc/oryx_127.0.206.66.keytab
--principal=oryx/127.0.206.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.206.66:38513
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781864224577058-825-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.206.66
--webserver_port=33079
--tserver_master_addrs=127.0.206.126:45539
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9671
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.206.66
--log_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781864224577058-825-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 09:32:44 UTC on ccycloud.quasar-gbtldp.root.comops.site
build id 4455440
I20260619 10:17:41.489642 9671 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 10:17:41.489866 9671 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 10:17:41.494596 9677 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 10:17:41.494624 9676 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 10:17:41.494675 9679 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 10:17:41.494798 9671 server_base.cc:1034] running on GCE node
I20260619 10:17:41.494969 9671 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 10:17:41.495157 9671 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 10:17:41.496292 9671 hybrid_clock.cc:648] HybridClock initialized: now 1781864261496279 us; error 28 us; skew 500 ppm
Jun 19 10:17:41 dist-test-slave-6pt2 krb5kdc[8935](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864261, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.206.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 10:17:41.498776 9671 init.cc:377] Logged in from keytab as oryx/127.0.206.66@KRBTEST.COM (short username oryx)
I20260619 10:17:41.499200 9671 webserver.cc:460] Webserver started at http://127.0.206.66:33079/ using document root <none> and password file <none>
I20260619 10:17:41.499405 9671 fs_manager.cc:359] Metadata directory not provided
I20260619 10:17:41.499454 9671 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 10:17:41.500675 9671 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 10:17:41.501173 9686 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 10:17:41.501389 9671 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 10:17:41.501451 9671 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781864224577058-825-0/minicluster-data/ts-1/data,/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781864224577058-825-0/minicluster-data/ts-1/wal
uuid: "5c2d08d0ab8e4118885f8b5209af023b"
format_stamp: "Formatted at 2026-06-19 10:17:40 on dist-test-slave-6pt2"
server_key: "b57e11c2f41bce8c18d1d2dd6bb0b4e8"
server_key_iv: "3e5bcb2c8062c1afca745a74693f6acc"
server_key_version: "encryptionkey@0"
I20260619 10:17:41.501755 9671 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781864224577058-825-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781864224577058-825-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781864224577058-825-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 10:17:41.521551 9671 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 10:17:41.523346 9671 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 10:17:41.523509 9671 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 10:17:41.523729 9671 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 10:17:41.524029 9671 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 10:17:41.524062 9671 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 10:17:41.524083 9671 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 10:17:41.524128 9671 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 10:17:41.529855 9671 rpc_server.cc:307] RPC server started. Bound to: 127.0.206.66:38513
I20260619 10:17:41.529933 9799 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.206.66:38513 every 8 connection(s)
I20260619 10:17:41.530243 9671 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781864224577058-825-0/minicluster-data/ts-1/data/info.pb
I20260619 10:17:41.530529 825 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTU9OP6/build/release/bin/kudu as pid 9671
I20260619 10:17:41.530836 825 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTU9OP6/build/release/bin/kudu
/tmp/dist-test-taskTU9OP6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781864224577058-825-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781864224577058-825-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781864224577058-825-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781864224577058-825-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.206.67:36771
--local_ip_for_outbound_sockets=127.0.206.67
--tserver_master_addrs=127.0.206.126:45539
--webserver_port=42637
--webserver_interface=127.0.206.67
--keytab_file=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781864224577058-825-0/krb5kdc/kudu_127.0.206.67.keytab
--principal=kudu/127.0.206.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.206.84:38663
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.0.206.67
--keytab_file=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781864224577058-825-0/krb5kdc/oryx_127.0.206.67.keytab with env {KRB5CCNAME=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781864224577058-825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781864224577058-825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781864224577058-825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Jun 19 10:17:41 dist-test-slave-6pt2 krb5kdc[8935](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.206.66@KRBTEST.COM for oryx/127.0.206.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 10:17:41 dist-test-slave-6pt2 krb5kdc[8935](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.206.66@KRBTEST.COM for oryx/127.0.206.126@KRBTEST.COM, Server not found in Kerberos database
W20260619 10:17:41.538980 9669 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 10:17:41.530954 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.66:42115 (local address 127.0.206.126:45539)
0619 10:17:41.531001 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0619 10:17:41.531003 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 10:17:41.531318 (+ 315us) server_negotiation.cc:407] Connection header received
0619 10:17:41.532155 (+ 837us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:17:41.532157 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:17:41.532177 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:17:41.532254 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:17:41.533197 (+ 943us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:41.533678 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:41.535364 (+ 1686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:41.535544 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:41.535596 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:41.538926 (+ 3330us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.206.66:42115: BlockingRecv error: recv got EOF from 127.0.206.66:42115 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
I20260619 10:17:41.539033 9802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:41.530946 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:45539 (local address 127.0.206.66:42115)
0619 10:17:41.531165 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0619 10:17:41.531211 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0619 10:17:41.532028 (+ 817us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 10:17:41.532328 (+ 300us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 10:17:41.532332 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 10:17:41.532867 (+ 535us) client_negotiation.cc:189] Negotiated authn=SASL
0619 10:17:41.533074 (+ 207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:41.533080 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:41.533775 (+ 695us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:41.533776 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:41.535264 (+ 1488us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:41.535267 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:41.535621 (+ 354us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:41.535623 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:41.535766 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:41.536264 (+ 498us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 10:17:41.536274 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0619 10:17:41.538852 (+ 2578us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.206.126:45539: Server oryx/127.0.206.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":179,"thread_start_us":88,"threads_started":1}
W20260619 10:17:41.539254 9800 heartbeater.cc:644] Failed to heartbeat to 127.0.206.126:45539 (0 consecutive failures): Not authorized: Failed to ping master at 127.0.206.126:45539: Client connection negotiation failed: client connection to 127.0.206.126:45539: Server oryx/127.0.206.126@KRBTEST.COM not found in Kerberos database
Jun 19 10:17:41 dist-test-slave-6pt2 krb5kdc[8935](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.206.66@KRBTEST.COM for oryx/127.0.206.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 10:17:41 dist-test-slave-6pt2 krb5kdc[8935](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.206.66@KRBTEST.COM for oryx/127.0.206.126@KRBTEST.COM, Server not found in Kerberos database
I20260619 10:17:41.543304 9802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:41.539464 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:45539 (local address 127.0.206.66:39099)
0619 10:17:41.539514 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0619 10:17:41.539525 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 10:17:41.539561 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 10:17:41.539697 (+ 136us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 10:17:41.539697 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 10:17:41.539818 (+ 121us) client_negotiation.cc:189] Negotiated authn=SASL
0619 10:17:41.539856 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:41.539857 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:41.540447 (+ 590us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:41.540448 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:41.540918 (+ 470us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:41.540919 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:41.541253 (+ 334us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:41.541254 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:41.541324 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:41.541717 (+ 393us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 10:17:41.541721 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 10:17:41.543247 (+ 1526us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.206.126:45539: Server oryx/127.0.206.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":35}
W20260619 10:17:41.543377 9669 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 10:17:41.539528 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.66:39099 (local address 127.0.206.126:45539)
0619 10:17:41.539574 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0619 10:17:41.539576 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 10:17:41.539584 (+ 8us) server_negotiation.cc:407] Connection header received
0619 10:17:41.539616 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:17:41.539617 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:17:41.539625 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:17:41.539693 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:17:41.539938 (+ 245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:41.540362 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:41.541003 (+ 641us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:41.541176 (+ 173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:41.541253 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:41.543315 (+ 2062us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.206.66:39099: BlockingRecv error: recv got EOF from 127.0.206.66:39099 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
Jun 19 10:17:41 dist-test-slave-6pt2 krb5kdc[8935](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.206.66@KRBTEST.COM for oryx/127.0.206.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 10:17:41 dist-test-slave-6pt2 krb5kdc[8935](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.206.66@KRBTEST.COM for oryx/127.0.206.126@KRBTEST.COM, Server not found in Kerberos database
I20260619 10:17:41.547596 9802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:41.543659 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:45539 (local address 127.0.206.66:51369)
0619 10:17:41.543703 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0619 10:17:41.543713 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 10:17:41.543769 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 10:17:41.543941 (+ 172us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 10:17:41.543942 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 10:17:41.544059 (+ 117us) client_negotiation.cc:189] Negotiated authn=SASL
0619 10:17:41.544082 (+ 23us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:41.544084 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:41.544734 (+ 650us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:41.544735 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:41.545212 (+ 477us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:41.545213 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:41.545565 (+ 352us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:41.545566 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:41.545614 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:41.546001 (+ 387us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 10:17:41.546005 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 10:17:41.547526 (+ 1521us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.206.126:45539: Server oryx/127.0.206.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":30}
W20260619 10:17:41.547646 9669 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 10:17:41.543720 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.66:51369 (local address 127.0.206.126:45539)
0619 10:17:41.543772 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0619 10:17:41.543774 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 10:17:41.543782 (+ 8us) server_negotiation.cc:407] Connection header received
0619 10:17:41.543852 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:17:41.543853 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:17:41.543861 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:17:41.543938 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:17:41.544161 (+ 223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:41.544661 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:41.545303 (+ 642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:41.545487 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:41.545564 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:41.547587 (+ 2023us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.206.66:51369: BlockingRecv error: recv got EOF from 127.0.206.66:51369 (error 108)
Metrics: {"server-negotiator.queue_time_us":35}
W20260619 10:17:41.547793 9800 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260619 10:17:41.611873 9803 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 10:17:41.612085 9803 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 10:17:41.612107 9803 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 10:17:41.613174 9803 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 10:17:41.613214 9803 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 10:17:41.613238 9803 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.206.67
I20260619 10:17:41.614707 9803 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.206.84:38663
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781864224577058-825-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781864224577058-825-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781864224577058-825-0/krb5kdc/oryx_127.0.206.67.keytab
--principal=oryx/127.0.206.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.206.67:36771
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781864224577058-825-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.206.67
--webserver_port=42637
--tserver_master_addrs=127.0.206.126:45539
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9803
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.206.67
--log_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781864224577058-825-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 09:32:44 UTC on ccycloud.quasar-gbtldp.root.comops.site
build id 4455440
I20260619 10:17:41.614931 9803 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 10:17:41.615162 9803 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 10:17:41.619737 9811 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 10:17:41.619803 9809 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 10:17:41.619747 9808 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 10:17:41.619932 9803 server_base.cc:1034] running on GCE node
I20260619 10:17:41.620158 9803 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 10:17:41.620373 9803 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 10:17:41.621501 9803 hybrid_clock.cc:648] HybridClock initialized: now 1781864261621502 us; error 38 us; skew 500 ppm
Jun 19 10:17:41 dist-test-slave-6pt2 krb5kdc[8935](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864261, etypes {rep=17 tkt=17 ses=17}, oryx/127.0.206.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 10:17:41.624037 9803 init.cc:377] Logged in from keytab as oryx/127.0.206.67@KRBTEST.COM (short username oryx)
I20260619 10:17:41.624409 9803 webserver.cc:460] Webserver started at http://127.0.206.67:42637/ using document root <none> and password file <none>
I20260619 10:17:41.624620 9803 fs_manager.cc:359] Metadata directory not provided
I20260619 10:17:41.624696 9803 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 10:17:41.625926 9803 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 10:17:41.626397 9818 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 10:17:41.626538 9803 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260619 10:17:41.626652 9803 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781864224577058-825-0/minicluster-data/ts-2/data,/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781864224577058-825-0/minicluster-data/ts-2/wal
uuid: "d5aac998d97c49a1bbf6bfe55ba25065"
format_stamp: "Formatted at 2026-06-19 10:17:41 on dist-test-slave-6pt2"
server_key: "8473c3cddefdd8bebbc02ab0285a3d78"
server_key_iv: "0dbe17ddeade59498194bfde36a10171"
server_key_version: "encryptionkey@0"
I20260619 10:17:41.626961 9803 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781864224577058-825-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781864224577058-825-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781864224577058-825-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 10:17:41.638527 9803 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 10:17:41.640690 9803 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 10:17:41.640859 9803 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 10:17:41.641055 9803 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 10:17:41.641330 9803 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 10:17:41.641381 9803 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 10:17:41.641418 9803 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 10:17:41.641446 9803 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 10:17:41.646819 9803 rpc_server.cc:307] RPC server started. Bound to: 127.0.206.67:36771
I20260619 10:17:41.646872 9931 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.206.67:36771 every 8 connection(s)
I20260619 10:17:41.647164 9803 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781864224577058-825-0/minicluster-data/ts-2/data/info.pb
Jun 19 10:17:41 dist-test-slave-6pt2 krb5kdc[8935](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.206.67@KRBTEST.COM for oryx/127.0.206.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 10:17:41 dist-test-slave-6pt2 krb5kdc[8935](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.206.67@KRBTEST.COM for oryx/127.0.206.126@KRBTEST.COM, Server not found in Kerberos database
W20260619 10:17:41.653702 9669 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 10:17:41.647777 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.67:46369 (local address 127.0.206.126:45539)
0619 10:17:41.647834 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0619 10:17:41.647836 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 10:17:41.648009 (+ 173us) server_negotiation.cc:407] Connection header received
0619 10:17:41.648496 (+ 487us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:17:41.648497 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:17:41.648509 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:17:41.648562 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:17:41.649435 (+ 873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:41.649911 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:41.650902 (+ 991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:41.651099 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:41.651170 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:41.653655 (+ 2485us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.206.67:46369: BlockingRecv error: recv got EOF from 127.0.206.67:46369 (error 108)
Metrics: {"server-negotiator.queue_time_us":32}
I20260619 10:17:41.653762 9934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:41.647759 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:45539 (local address 127.0.206.67:46369)
0619 10:17:41.647901 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0619 10:17:41.647933 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0619 10:17:41.648396 (+ 463us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 10:17:41.648630 (+ 234us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 10:17:41.648633 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 10:17:41.649160 (+ 527us) client_negotiation.cc:189] Negotiated authn=SASL
0619 10:17:41.649325 (+ 165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:41.649330 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:41.650000 (+ 670us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:41.650001 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:41.650809 (+ 808us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:41.650811 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:41.651174 (+ 363us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:41.651175 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:41.651294 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:41.651650 (+ 356us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 10:17:41.651657 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0619 10:17:41.653582 (+ 1925us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.206.126:45539: Server oryx/127.0.206.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":70,"threads_started":1}
W20260619 10:17:41.653959 9932 heartbeater.cc:644] Failed to heartbeat to 127.0.206.126:45539 (0 consecutive failures): Not authorized: Failed to ping master at 127.0.206.126:45539: Client connection negotiation failed: client connection to 127.0.206.126:45539: Server oryx/127.0.206.126@KRBTEST.COM not found in Kerberos database
I20260619 10:17:41.655761 825 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTU9OP6/build/release/bin/kudu as pid 9803
Jun 19 10:17:41 dist-test-slave-6pt2 krb5kdc[8935](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.206.67@KRBTEST.COM for oryx/127.0.206.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 10:17:41 dist-test-slave-6pt2 krb5kdc[8935](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.206.67@KRBTEST.COM for oryx/127.0.206.126@KRBTEST.COM, Server not found in Kerberos database
I20260619 10:17:41.658761 9934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:41.654190 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:45539 (local address 127.0.206.67:47263)
0619 10:17:41.654235 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0619 10:17:41.654245 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 10:17:41.654279 (+ 34us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 10:17:41.654472 (+ 193us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 10:17:41.654473 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 10:17:41.654574 (+ 101us) client_negotiation.cc:189] Negotiated authn=SASL
0619 10:17:41.654662 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:41.654663 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:41.655213 (+ 550us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:41.655213 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:41.655880 (+ 667us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:41.655883 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:41.656331 (+ 448us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:41.656332 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:41.656414 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:41.656773 (+ 359us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 10:17:41.656778 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 10:17:41.658708 (+ 1930us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.206.126:45539: Server oryx/127.0.206.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":31}
W20260619 10:17:41.658776 9669 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 10:17:41.654330 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.67:47263 (local address 127.0.206.126:45539)
0619 10:17:41.654381 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0619 10:17:41.654383 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 10:17:41.654390 (+ 7us) server_negotiation.cc:407] Connection header received
0619 10:17:41.654406 (+ 16us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:17:41.654406 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:17:41.654412 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:17:41.654459 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:17:41.654721 (+ 262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:41.655152 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:41.655960 (+ 808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:41.656256 (+ 296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:41.656319 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:41.658725 (+ 2406us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.206.67:47263: BlockingRecv error: recv got EOF from 127.0.206.67:47263 (error 108)
Metrics: {"server-negotiator.queue_time_us":34}
Jun 19 10:17:41 dist-test-slave-6pt2 krb5kdc[8935](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.206.67@KRBTEST.COM for oryx/127.0.206.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 10:17:41 dist-test-slave-6pt2 krb5kdc[8935](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.206.67@KRBTEST.COM for oryx/127.0.206.126@KRBTEST.COM, Server not found in Kerberos database
W20260619 10:17:41.664105 9669 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 10:17:41.659242 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.67:57335 (local address 127.0.206.126:45539)
0619 10:17:41.659275 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0619 10:17:41.659277 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 10:17:41.659285 (+ 8us) server_negotiation.cc:407] Connection header received
0619 10:17:41.659336 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:17:41.659337 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:17:41.659345 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:17:41.659391 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:17:41.659694 (+ 303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:41.660227 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:41.661135 (+ 908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:41.661416 (+ 281us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:41.661499 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:41.664057 (+ 2558us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.206.67:57335: BlockingRecv error: recv got EOF from 127.0.206.67:57335 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
I20260619 10:17:41.664271 9934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:41.659182 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:45539 (local address 127.0.206.67:57335)
0619 10:17:41.659232 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0619 10:17:41.659242 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 10:17:41.659278 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 10:17:41.659430 (+ 152us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 10:17:41.659431 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 10:17:41.659584 (+ 153us) client_negotiation.cc:189] Negotiated authn=SASL
0619 10:17:41.659611 (+ 27us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:41.659613 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:41.660305 (+ 692us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:41.660306 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:41.661046 (+ 740us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:41.661049 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:41.661522 (+ 473us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:41.661523 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:41.661597 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:41.661994 (+ 397us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 10:17:41.661999 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 10:17:41.664210 (+ 2211us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.206.126:45539: Server oryx/127.0.206.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":34}
I20260619 10:17:41.664331 9670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:41.656174 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36258 (local address 127.0.206.126:45539)
0619 10:17:41.656236 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0619 10:17:41.656238 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 10:17:41.656392 (+ 154us) server_negotiation.cc:407] Connection header received
0619 10:17:41.656473 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:17:41.656473 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:17:41.656481 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:17:41.656575 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:17:41.657327 (+ 752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:41.658046 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:41.659110 (+ 1064us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:41.659381 (+ 271us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:41.659429 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:41.660696 (+ 1267us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 10:17:41.660700 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 10:17:41.660700 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 10:17:41.660711 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0619 10:17:41.662781 (+ 2070us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:17:41.663182 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:17:41.663185 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:17:41.663185 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:17:41.663297 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:17:41.663510 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:17:41.663511 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:17:41.663511 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:17:41.663864 (+ 353us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 10:17:41.664010 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0619 10:17:41.664123 (+ 113us) server_negotiation.cc:299] Negotiation successful
0619 10:17:41.664238 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
W20260619 10:17:41.664379 9932 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Jun 19 10:17:42 dist-test-slave-6pt2 krb5kdc[8935](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.206.65@KRBTEST.COM for oryx/127.0.206.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 10:17:42 dist-test-slave-6pt2 krb5kdc[8935](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.206.65@KRBTEST.COM for oryx/127.0.206.126@KRBTEST.COM, Server not found in Kerberos database
I20260619 10:17:42.420213 9936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:42.415835 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:45539 (local address 127.0.206.65:56179)
0619 10:17:42.415979 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0619 10:17:42.415989 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 10:17:42.416034 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 10:17:42.416193 (+ 159us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 10:17:42.416194 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 10:17:42.416350 (+ 156us) client_negotiation.cc:189] Negotiated authn=SASL
0619 10:17:42.416409 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:42.416410 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:42.417058 (+ 648us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:42.417059 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:42.417593 (+ 534us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:42.417594 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:42.417900 (+ 306us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:42.417901 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:42.417945 (+ 44us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:42.418345 (+ 400us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 10:17:42.418350 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 10:17:42.420152 (+ 1802us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.206.126:45539: Server oryx/127.0.206.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":79,"threads_started":1}
W20260619 10:17:42.420303 9937 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 10:17:42.415903 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.65:56179 (local address 127.0.206.126:45539)
0619 10:17:42.416030 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0619 10:17:42.416032 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 10:17:42.416040 (+ 8us) server_negotiation.cc:407] Connection header received
0619 10:17:42.416093 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:17:42.416094 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:17:42.416102 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:17:42.416151 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:17:42.416470 (+ 319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:42.416980 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:42.417660 (+ 680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:42.417842 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:42.417889 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:42.420234 (+ 2345us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.206.65:56179: BlockingRecv error: recv got EOF from 127.0.206.65:56179 (error 108)
Metrics: {"server-negotiator.queue_time_us":107,"thread_start_us":79,"threads_started":1}
Jun 19 10:17:42 dist-test-slave-6pt2 krb5kdc[8935](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.206.66@KRBTEST.COM for oryx/127.0.206.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 10:17:42 dist-test-slave-6pt2 krb5kdc[8935](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.206.66@KRBTEST.COM for oryx/127.0.206.126@KRBTEST.COM, Server not found in Kerberos database
I20260619 10:17:42.552981 9938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:42.548212 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:45539 (local address 127.0.206.66:40735)
0619 10:17:42.548376 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0619 10:17:42.548388 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 10:17:42.548462 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 10:17:42.548586 (+ 124us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 10:17:42.548587 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 10:17:42.548755 (+ 168us) client_negotiation.cc:189] Negotiated authn=SASL
0619 10:17:42.548799 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:42.548801 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:42.549530 (+ 729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:42.549531 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:42.550076 (+ 545us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:42.550078 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:42.550485 (+ 407us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:42.550486 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:42.550548 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:42.551119 (+ 571us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 10:17:42.551125 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0619 10:17:42.552860 (+ 1735us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.206.126:45539: Server oryx/127.0.206.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":90,"threads_started":1}
W20260619 10:17:42.553012 9937 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 10:17:42.548307 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.66:40735 (local address 127.0.206.126:45539)
0619 10:17:42.548391 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0619 10:17:42.548392 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 10:17:42.548466 (+ 74us) server_negotiation.cc:407] Connection header received
0619 10:17:42.548518 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:17:42.548518 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:17:42.548530 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:17:42.548582 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:17:42.548909 (+ 327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:42.549444 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:42.550164 (+ 720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:42.550368 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:42.550483 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:42.552939 (+ 2456us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.206.66:40735: BlockingRecv error: recv got EOF from 127.0.206.66:40735 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
Jun 19 10:17:42 dist-test-slave-6pt2 krb5kdc[8935](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.206.67@KRBTEST.COM for oryx/127.0.206.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 10:17:42 dist-test-slave-6pt2 krb5kdc[8935](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.206.67@KRBTEST.COM for oryx/127.0.206.126@KRBTEST.COM, Server not found in Kerberos database
I20260619 10:17:42.669868 9939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:42.664716 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:45539 (local address 127.0.206.67:39677)
0619 10:17:42.664824 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0619 10:17:42.664835 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 10:17:42.664948 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 10:17:42.665103 (+ 155us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 10:17:42.665104 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 10:17:42.665289 (+ 185us) client_negotiation.cc:189] Negotiated authn=SASL
0619 10:17:42.665392 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:42.665395 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:42.666389 (+ 994us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:42.666390 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:42.667063 (+ 673us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:42.667066 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:42.667554 (+ 488us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:42.667555 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:42.667613 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:42.668055 (+ 442us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 10:17:42.668060 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 10:17:42.669756 (+ 1696us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.206.126:45539: Server oryx/127.0.206.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":89,"thread_start_us":64,"threads_started":1}
W20260619 10:17:42.669890 9937 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 10:17:42.664793 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.67:39677 (local address 127.0.206.126:45539)
0619 10:17:42.664839 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0619 10:17:42.664842 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 10:17:42.664945 (+ 103us) server_negotiation.cc:407] Connection header received
0619 10:17:42.665033 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:17:42.665034 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:17:42.665047 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:17:42.665099 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:17:42.665480 (+ 381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:42.666276 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:42.667164 (+ 888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:42.667453 (+ 289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:42.667541 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:42.669829 (+ 2288us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.206.67:39677: BlockingRecv error: recv got EOF from 127.0.206.67:39677 (error 108)
Metrics: {"server-negotiator.queue_time_us":24}
Jun 19 10:17:43 dist-test-slave-6pt2 krb5kdc[8935](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.206.65@KRBTEST.COM for oryx/127.0.206.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 10:17:43 dist-test-slave-6pt2 krb5kdc[8935](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.206.65@KRBTEST.COM for oryx/127.0.206.126@KRBTEST.COM, Server not found in Kerberos database
I20260619 10:17:43.425275 9940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:43.420810 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:45539 (local address 127.0.206.65:49437)
0619 10:17:43.420990 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0619 10:17:43.421004 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0619 10:17:43.421085 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 10:17:43.421253 (+ 168us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 10:17:43.421253 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 10:17:43.421497 (+ 244us) client_negotiation.cc:189] Negotiated authn=SASL
0619 10:17:43.421556 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:43.421558 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:43.422209 (+ 651us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:43.422210 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:43.422741 (+ 531us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:43.422743 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:43.423053 (+ 310us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:43.423054 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:43.423090 (+ 36us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:43.423565 (+ 475us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 10:17:43.423571 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0619 10:17:43.425218 (+ 1647us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.206.126:45539: Server oryx/127.0.206.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":161,"thread_start_us":110,"threads_started":1}
W20260619 10:17:43.425398 9941 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 10:17:43.420916 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.65:49437 (local address 127.0.206.126:45539)
0619 10:17:43.421040 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0619 10:17:43.421042 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 10:17:43.421099 (+ 57us) server_negotiation.cc:407] Connection header received
0619 10:17:43.421131 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:17:43.421132 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:17:43.421144 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:17:43.421236 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:17:43.421631 (+ 395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:43.422145 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:43.422801 (+ 656us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:43.422993 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:43.423044 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:43.425300 (+ 2256us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.206.65:49437: BlockingRecv error: recv got EOF from 127.0.206.65:49437 (error 108)
Metrics: {"server-negotiator.queue_time_us":105,"thread_start_us":65,"threads_started":1}
Jun 19 10:17:43 dist-test-slave-6pt2 krb5kdc[8935](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.206.66@KRBTEST.COM for oryx/127.0.206.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 10:17:43 dist-test-slave-6pt2 krb5kdc[8935](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.206.66@KRBTEST.COM for oryx/127.0.206.126@KRBTEST.COM, Server not found in Kerberos database
I20260619 10:17:43.557919 9942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:43.553527 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:45539 (local address 127.0.206.66:38131)
0619 10:17:43.553655 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0619 10:17:43.553665 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 10:17:43.553718 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 10:17:43.553834 (+ 116us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 10:17:43.553834 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 10:17:43.553991 (+ 157us) client_negotiation.cc:189] Negotiated authn=SASL
0619 10:17:43.554037 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:43.554039 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:43.554729 (+ 690us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:43.554730 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:43.555262 (+ 532us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:43.555263 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:43.555633 (+ 370us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:43.555634 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:43.555697 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:43.556116 (+ 419us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 10:17:43.556120 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 10:17:43.557829 (+ 1709us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.206.126:45539: Server oryx/127.0.206.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":72,"threads_started":1}
W20260619 10:17:43.557966 9941 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 10:17:43.553598 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.66:38131 (local address 127.0.206.126:45539)
0619 10:17:43.553652 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0619 10:17:43.553654 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 10:17:43.553703 (+ 49us) server_negotiation.cc:407] Connection header received
0619 10:17:43.553758 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:17:43.553759 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:17:43.553768 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:17:43.553814 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:17:43.554098 (+ 284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:43.554630 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:43.555346 (+ 716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:43.555550 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:43.555629 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:43.557906 (+ 2277us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.206.66:38131: BlockingRecv error: recv got EOF from 127.0.206.66:38131 (error 108)
Metrics: {"server-negotiator.queue_time_us":30}
Jun 19 10:17:43 dist-test-slave-6pt2 krb5kdc[8935](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.206.67@KRBTEST.COM for oryx/127.0.206.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 10:17:43 dist-test-slave-6pt2 krb5kdc[8935](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.206.67@KRBTEST.COM for oryx/127.0.206.126@KRBTEST.COM, Server not found in Kerberos database
I20260619 10:17:43.675207 9943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:43.670282 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:45539 (local address 127.0.206.67:55635)
0619 10:17:43.670417 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0619 10:17:43.670429 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 10:17:43.670480 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 10:17:43.671049 (+ 569us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 10:17:43.671050 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 10:17:43.671214 (+ 164us) client_negotiation.cc:189] Negotiated authn=SASL
0619 10:17:43.671261 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:43.671263 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:43.672017 (+ 754us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:43.672018 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:43.672565 (+ 547us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:43.672567 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:43.672954 (+ 387us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:43.672955 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:43.673026 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:43.673438 (+ 412us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 10:17:43.673442 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 10:17:43.675151 (+ 1709us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.206.126:45539: Server oryx/127.0.206.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":82,"threads_started":1}
W20260619 10:17:43.675338 9941 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 10:17:43.670372 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.67:55635 (local address 127.0.206.126:45539)
0619 10:17:43.670905 (+ 533us) server_negotiation.cc:206] Beginning negotiation
0619 10:17:43.670907 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 10:17:43.670916 (+ 9us) server_negotiation.cc:407] Connection header received
0619 10:17:43.670943 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:17:43.670944 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:17:43.670959 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:17:43.671026 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:17:43.671353 (+ 327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:43.671925 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:43.672662 (+ 737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:43.672870 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:43.672953 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:43.675256 (+ 2303us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.206.67:55635: BlockingRecv error: recv got EOF from 127.0.206.67:55635 (error 108)
Metrics: {"server-negotiator.queue_time_us":508}
Jun 19 10:17:44 dist-test-slave-6pt2 krb5kdc[8935](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.206.65@KRBTEST.COM for oryx/127.0.206.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 10:17:44 dist-test-slave-6pt2 krb5kdc[8935](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.206.65@KRBTEST.COM for oryx/127.0.206.126@KRBTEST.COM, Server not found in Kerberos database
I20260619 10:17:44.430265 9944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:44.425802 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:45539 (local address 127.0.206.65:45859)
0619 10:17:44.425951 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0619 10:17:44.425964 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 10:17:44.426038 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 10:17:44.426187 (+ 149us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 10:17:44.426188 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 10:17:44.426364 (+ 176us) client_negotiation.cc:189] Negotiated authn=SASL
0619 10:17:44.426412 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:44.426414 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:44.427071 (+ 657us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:44.427072 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:44.427622 (+ 550us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:44.427624 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:44.427947 (+ 323us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:44.427948 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:44.427986 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:44.428473 (+ 487us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 10:17:44.428477 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 10:17:44.430200 (+ 1723us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.206.126:45539: Server oryx/127.0.206.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":69,"threads_started":1}
W20260619 10:17:44.430379 9945 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 10:17:44.425906 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.65:45859 (local address 127.0.206.126:45539)
0619 10:17:44.426033 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0619 10:17:44.426034 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 10:17:44.426043 (+ 9us) server_negotiation.cc:407] Connection header received
0619 10:17:44.426081 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:17:44.426082 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:17:44.426092 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:17:44.426160 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:17:44.426476 (+ 316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:44.427002 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:44.427693 (+ 691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:44.427874 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:44.427921 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:44.430306 (+ 2385us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.206.65:45859: BlockingRecv error: recv got EOF from 127.0.206.65:45859 (error 108)
Metrics: {"server-negotiator.queue_time_us":107,"thread_start_us":63,"threads_started":1}
Jun 19 10:17:44 dist-test-slave-6pt2 krb5kdc[8935](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.206.66@KRBTEST.COM for oryx/127.0.206.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 10:17:44 dist-test-slave-6pt2 krb5kdc[8935](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.206.66@KRBTEST.COM for oryx/127.0.206.126@KRBTEST.COM, Server not found in Kerberos database
I20260619 10:17:44.562960 9946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:44.558360 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:45539 (local address 127.0.206.66:42245)
0619 10:17:44.558547 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0619 10:17:44.558558 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 10:17:44.558663 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 10:17:44.558804 (+ 141us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 10:17:44.558805 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 10:17:44.558954 (+ 149us) client_negotiation.cc:189] Negotiated authn=SASL
0619 10:17:44.559000 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:44.559002 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:44.559665 (+ 663us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:44.559666 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:44.560169 (+ 503us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:44.560170 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:44.560458 (+ 288us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:44.560459 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:44.560493 (+ 34us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:44.561014 (+ 521us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 10:17:44.561020 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0619 10:17:44.562857 (+ 1837us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.206.126:45539: Server oryx/127.0.206.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":97,"threads_started":1}
W20260619 10:17:44.563000 9945 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 10:17:44.558440 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.66:42245 (local address 127.0.206.126:45539)
0619 10:17:44.558503 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0619 10:17:44.558504 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 10:17:44.558634 (+ 130us) server_negotiation.cc:407] Connection header received
0619 10:17:44.558735 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:17:44.558736 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:17:44.558746 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:17:44.558785 (+ 39us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:17:44.559057 (+ 272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:44.559604 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:44.560230 (+ 626us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:44.560400 (+ 170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:44.560446 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:44.562929 (+ 2483us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.206.66:42245: BlockingRecv error: recv got EOF from 127.0.206.66:42245 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
Jun 19 10:17:44 dist-test-slave-6pt2 krb5kdc[8935](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.206.67@KRBTEST.COM for oryx/127.0.206.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 10:17:44 dist-test-slave-6pt2 krb5kdc[8935](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.206.67@KRBTEST.COM for oryx/127.0.206.126@KRBTEST.COM, Server not found in Kerberos database
I20260619 10:17:44.680035 9947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:44.675627 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:45539 (local address 127.0.206.67:49251)
0619 10:17:44.675795 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0619 10:17:44.675804 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0619 10:17:44.675886 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 10:17:44.676022 (+ 136us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 10:17:44.676023 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 10:17:44.676170 (+ 147us) client_negotiation.cc:189] Negotiated authn=SASL
0619 10:17:44.676215 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:44.676216 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:44.676846 (+ 630us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:44.676847 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:44.677407 (+ 560us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:44.677409 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:44.677690 (+ 281us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:44.677691 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:44.677749 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:44.678298 (+ 549us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 10:17:44.678302 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 10:17:44.679969 (+ 1667us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.206.126:45539: Server oryx/127.0.206.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":81,"threads_started":1}
W20260619 10:17:44.680140 9945 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 10:17:44.675690 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.67:49251 (local address 127.0.206.126:45539)
0619 10:17:44.675748 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0619 10:17:44.675749 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 10:17:44.675863 (+ 114us) server_negotiation.cc:407] Connection header received
0619 10:17:44.675949 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:17:44.675949 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:17:44.675960 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:17:44.676003 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:17:44.676294 (+ 291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:44.676782 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:44.677459 (+ 677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:44.677639 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:44.677689 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:44.680049 (+ 2360us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.206.67:49251: BlockingRecv error: recv got EOF from 127.0.206.67:49251 (error 108)
Metrics: {"server-negotiator.queue_time_us":36}
Jun 19 10:17:45 dist-test-slave-6pt2 krb5kdc[8935](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.206.65@KRBTEST.COM for oryx/127.0.206.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 10:17:45 dist-test-slave-6pt2 krb5kdc[8935](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.206.65@KRBTEST.COM for oryx/127.0.206.126@KRBTEST.COM, Server not found in Kerberos database
I20260619 10:17:45.435133 9948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:45.430720 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:45539 (local address 127.0.206.65:52437)
0619 10:17:45.430886 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0619 10:17:45.430897 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 10:17:45.430972 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 10:17:45.431100 (+ 128us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 10:17:45.431101 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 10:17:45.431282 (+ 181us) client_negotiation.cc:189] Negotiated authn=SASL
0619 10:17:45.431329 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:45.431331 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:45.431966 (+ 635us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:45.431966 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:45.432493 (+ 527us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:45.432494 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:45.432806 (+ 312us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:45.432807 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:45.432843 (+ 36us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:45.433268 (+ 425us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 10:17:45.433272 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 10:17:45.435038 (+ 1766us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.206.126:45539: Server oryx/127.0.206.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":83,"threads_started":1}
W20260619 10:17:45.435181 9949 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 10:17:45.430771 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.65:52437 (local address 127.0.206.126:45539)
0619 10:17:45.430907 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0619 10:17:45.430908 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 10:17:45.430944 (+ 36us) server_negotiation.cc:407] Connection header received
0619 10:17:45.431017 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:17:45.431018 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:17:45.431027 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:17:45.431082 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:17:45.431392 (+ 310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:45.431907 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:45.432560 (+ 653us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:45.432748 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:45.432805 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:45.435112 (+ 2307us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.206.65:52437: BlockingRecv error: recv got EOF from 127.0.206.65:52437 (error 108)
Metrics: {"server-negotiator.queue_time_us":117,"thread_start_us":72,"threads_started":1}
Jun 19 10:17:45 dist-test-slave-6pt2 krb5kdc[8935](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.206.66@KRBTEST.COM for oryx/127.0.206.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 10:17:45 dist-test-slave-6pt2 krb5kdc[8935](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.206.66@KRBTEST.COM for oryx/127.0.206.126@KRBTEST.COM, Server not found in Kerberos database
I20260619 10:17:45.570504 9950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:45.563613 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:45539 (local address 127.0.206.66:35127)
0619 10:17:45.563745 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0619 10:17:45.563760 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0619 10:17:45.563813 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 10:17:45.563937 (+ 124us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 10:17:45.563938 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 10:17:45.564090 (+ 152us) client_negotiation.cc:189] Negotiated authn=SASL
0619 10:17:45.564139 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:45.564141 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:45.564759 (+ 618us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:45.564759 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:45.565260 (+ 501us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:45.565261 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:45.565591 (+ 330us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:45.565593 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:45.565662 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:45.568913 (+ 3251us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 10:17:45.568918 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 10:17:45.570452 (+ 1534us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.206.126:45539: Server oryx/127.0.206.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":79,"threads_started":1}
W20260619 10:17:45.570637 9949 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 10:17:45.563688 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.66:35127 (local address 127.0.206.126:45539)
0619 10:17:45.563750 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0619 10:17:45.563752 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 10:17:45.563801 (+ 49us) server_negotiation.cc:407] Connection header received
0619 10:17:45.563870 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:17:45.563870 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:17:45.563880 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:17:45.563920 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:17:45.564197 (+ 277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:45.564695 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:45.565326 (+ 631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:45.565500 (+ 174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:45.565588 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:45.570552 (+ 4964us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.206.66:35127: BlockingRecv error: recv got EOF from 127.0.206.66:35127 (error 108)
Metrics: {"server-negotiator.queue_time_us":37}
Jun 19 10:17:45 dist-test-slave-6pt2 krb5kdc[8935](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.206.67@KRBTEST.COM for oryx/127.0.206.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 10:17:45 dist-test-slave-6pt2 krb5kdc[8935](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.206.67@KRBTEST.COM for oryx/127.0.206.126@KRBTEST.COM, Server not found in Kerberos database
I20260619 10:17:45.684947 9951 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:45.680454 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:45539 (local address 127.0.206.67:40815)
0619 10:17:45.680636 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0619 10:17:45.680645 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0619 10:17:45.680699 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 10:17:45.680807 (+ 108us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 10:17:45.680807 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 10:17:45.680958 (+ 151us) client_negotiation.cc:189] Negotiated authn=SASL
0619 10:17:45.681023 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:45.681024 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:45.681629 (+ 605us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:45.681629 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:45.682165 (+ 536us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:45.682167 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:45.682464 (+ 297us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:45.682465 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:45.682501 (+ 36us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:45.683003 (+ 502us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 10:17:45.683014 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0619 10:17:45.684891 (+ 1877us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.206.126:45539: Server oryx/127.0.206.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":170,"thread_start_us":81,"threads_started":1}
W20260619 10:17:45.685040 9949 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 10:17:45.680503 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.67:40815 (local address 127.0.206.126:45539)
0619 10:17:45.680584 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0619 10:17:45.680586 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 10:17:45.680704 (+ 118us) server_negotiation.cc:407] Connection header received
0619 10:17:45.680734 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:17:45.680734 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:17:45.680746 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:17:45.680801 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:17:45.681080 (+ 279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:45.681564 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:45.682231 (+ 667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:45.682402 (+ 171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:45.682450 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:45.684973 (+ 2523us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.206.67:40815: BlockingRecv error: recv got EOF from 127.0.206.67:40815 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
Jun 19 10:17:46 dist-test-slave-6pt2 krb5kdc[8935](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.206.65@KRBTEST.COM for oryx/127.0.206.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 10:17:46 dist-test-slave-6pt2 krb5kdc[8935](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.206.65@KRBTEST.COM for oryx/127.0.206.126@KRBTEST.COM, Server not found in Kerberos database
I20260619 10:17:46.440024 9952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:46.435628 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:45539 (local address 127.0.206.65:33977)
0619 10:17:46.435785 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0619 10:17:46.435795 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 10:17:46.435846 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 10:17:46.435972 (+ 126us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 10:17:46.435973 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 10:17:46.436128 (+ 155us) client_negotiation.cc:189] Negotiated authn=SASL
0619 10:17:46.436174 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:46.436176 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:46.436821 (+ 645us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:46.436821 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:46.437326 (+ 505us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:46.437328 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:46.437820 (+ 492us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:46.437821 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:46.437881 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:46.438312 (+ 431us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 10:17:46.438316 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 10:17:46.439968 (+ 1652us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.206.126:45539: Server oryx/127.0.206.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":89,"threads_started":1}
W20260619 10:17:46.440155 9953 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 10:17:46.435707 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.65:33977 (local address 127.0.206.126:45539)
0619 10:17:46.435859 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0619 10:17:46.435861 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 10:17:46.435869 (+ 8us) server_negotiation.cc:407] Connection header received
0619 10:17:46.435889 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:17:46.435890 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:17:46.435899 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:17:46.435957 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:17:46.436240 (+ 283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:46.436754 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:46.437405 (+ 651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:46.437741 (+ 336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:46.437807 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:46.440051 (+ 2244us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.206.65:33977: BlockingRecv error: recv got EOF from 127.0.206.65:33977 (error 108)
Metrics: {"server-negotiator.queue_time_us":126,"thread_start_us":98,"threads_started":1}
Jun 19 10:17:46 dist-test-slave-6pt2 krb5kdc[8935](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.206.66@KRBTEST.COM for oryx/127.0.206.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 10:17:46 dist-test-slave-6pt2 krb5kdc[8935](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.206.66@KRBTEST.COM for oryx/127.0.206.126@KRBTEST.COM, Server not found in Kerberos database
I20260619 10:17:46.575387 9954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:46.570928 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:45539 (local address 127.0.206.66:59465)
0619 10:17:46.571049 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0619 10:17:46.571066 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0619 10:17:46.571113 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 10:17:46.571244 (+ 131us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 10:17:46.571244 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 10:17:46.571384 (+ 140us) client_negotiation.cc:189] Negotiated authn=SASL
0619 10:17:46.571429 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:46.571430 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:46.572051 (+ 621us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:46.572052 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:46.572593 (+ 541us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:46.572595 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:46.572923 (+ 328us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:46.572923 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:46.572963 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:46.573386 (+ 423us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 10:17:46.573390 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 10:17:46.575291 (+ 1901us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.206.126:45539: Server oryx/127.0.206.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":73,"threads_started":1}
W20260619 10:17:46.575435 9953 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 10:17:46.571047 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.66:59465 (local address 127.0.206.126:45539)
0619 10:17:46.571090 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0619 10:17:46.571091 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 10:17:46.571103 (+ 12us) server_negotiation.cc:407] Connection header received
0619 10:17:46.571156 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:17:46.571156 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:17:46.571165 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:17:46.571224 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:17:46.571486 (+ 262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:46.571969 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:46.572664 (+ 695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:46.572851 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:46.572911 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:46.575371 (+ 2460us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.206.66:59465: BlockingRecv error: recv got EOF from 127.0.206.66:59465 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
Jun 19 10:17:46 dist-test-slave-6pt2 krb5kdc[8935](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.206.67@KRBTEST.COM for oryx/127.0.206.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 10:17:46 dist-test-slave-6pt2 krb5kdc[8935](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.206.67@KRBTEST.COM for oryx/127.0.206.126@KRBTEST.COM, Server not found in Kerberos database
I20260619 10:17:46.690018 9955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:46.685399 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:45539 (local address 127.0.206.67:49747)
0619 10:17:46.685573 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0619 10:17:46.685585 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 10:17:46.685650 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 10:17:46.685793 (+ 143us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 10:17:46.685794 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 10:17:46.685944 (+ 150us) client_negotiation.cc:189] Negotiated authn=SASL
0619 10:17:46.685993 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:46.685994 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:46.686747 (+ 753us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:46.686748 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:46.687360 (+ 612us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:46.687362 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:46.687663 (+ 301us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:46.687664 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:46.687765 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:46.688244 (+ 479us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 10:17:46.688248 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 10:17:46.689956 (+ 1708us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.206.126:45539: Server oryx/127.0.206.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":79,"threads_started":1}
W20260619 10:17:46.690166 9953 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 10:17:46.685465 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.67:49747 (local address 127.0.206.126:45539)
0619 10:17:46.685558 (+ 93us) server_negotiation.cc:206] Beginning negotiation
0619 10:17:46.685560 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 10:17:46.685652 (+ 92us) server_negotiation.cc:407] Connection header received
0619 10:17:46.685704 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:17:46.685704 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:17:46.685719 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:17:46.685771 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:17:46.686052 (+ 281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:46.686643 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:46.687424 (+ 781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:46.687603 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:46.687654 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:46.690067 (+ 2413us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.206.67:49747: BlockingRecv error: recv got EOF from 127.0.206.67:49747 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
Jun 19 10:17:47 dist-test-slave-6pt2 krb5kdc[8935](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.206.65@KRBTEST.COM for oryx/127.0.206.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 10:17:47 dist-test-slave-6pt2 krb5kdc[8935](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.206.65@KRBTEST.COM for oryx/127.0.206.126@KRBTEST.COM, Server not found in Kerberos database
I20260619 10:17:47.444875 9956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:47.440461 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:45539 (local address 127.0.206.65:59883)
0619 10:17:47.440743 (+ 282us) negotiation.cc:107] Waiting for socket to connect
0619 10:17:47.440760 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0619 10:17:47.440816 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 10:17:47.440966 (+ 150us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 10:17:47.440966 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 10:17:47.441121 (+ 155us) client_negotiation.cc:189] Negotiated authn=SASL
0619 10:17:47.441168 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:47.441169 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:47.441775 (+ 606us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:47.441775 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:47.442265 (+ 490us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:47.442266 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:47.442604 (+ 338us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:47.442605 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:47.442658 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:47.443108 (+ 450us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 10:17:47.443113 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 10:17:47.444816 (+ 1703us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.206.126:45539: Server oryx/127.0.206.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":245,"thread_start_us":79,"threads_started":1}
W20260619 10:17:47.445042 9957 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 10:17:47.440742 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.65:59883 (local address 127.0.206.126:45539)
0619 10:17:47.440850 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0619 10:17:47.440852 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 10:17:47.440859 (+ 7us) server_negotiation.cc:407] Connection header received
0619 10:17:47.440879 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:17:47.440879 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:17:47.440888 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:17:47.440946 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:17:47.441231 (+ 285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:47.441714 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:47.442335 (+ 621us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:47.442525 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:47.442577 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:47.444934 (+ 2357us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.206.65:59883: BlockingRecv error: recv got EOF from 127.0.206.65:59883 (error 108)
Metrics: {"server-negotiator.queue_time_us":82,"thread_start_us":58,"threads_started":1}
Jun 19 10:17:47 dist-test-slave-6pt2 krb5kdc[8935](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.206.66@KRBTEST.COM for oryx/127.0.206.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 10:17:47 dist-test-slave-6pt2 krb5kdc[8935](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.206.66@KRBTEST.COM for oryx/127.0.206.126@KRBTEST.COM, Server not found in Kerberos database
I20260619 10:17:47.580571 9958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:47.575904 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:45539 (local address 127.0.206.66:56233)
0619 10:17:47.576071 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0619 10:17:47.576083 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 10:17:47.576138 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 10:17:47.576314 (+ 176us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 10:17:47.576315 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 10:17:47.576563 (+ 248us) client_negotiation.cc:189] Negotiated authn=SASL
0619 10:17:47.576630 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:47.576632 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:47.577331 (+ 699us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:47.577331 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:47.577883 (+ 552us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:47.577886 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:47.578191 (+ 305us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:47.578191 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:47.578258 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:47.578759 (+ 501us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 10:17:47.578764 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 10:17:47.580511 (+ 1747us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.206.126:45539: Server oryx/127.0.206.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":85,"threads_started":1}
W20260619 10:17:47.580686 9957 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 10:17:47.576008 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.66:56233 (local address 127.0.206.126:45539)
0619 10:17:47.576134 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0619 10:17:47.576136 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 10:17:47.576146 (+ 10us) server_negotiation.cc:407] Connection header received
0619 10:17:47.576194 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:17:47.576195 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:17:47.576209 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:17:47.576272 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:17:47.576707 (+ 435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:47.577253 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:47.577953 (+ 700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:47.578129 (+ 176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:47.578184 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:47.580595 (+ 2411us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.206.66:56233: BlockingRecv error: recv got EOF from 127.0.206.66:56233 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
Jun 19 10:17:47 dist-test-slave-6pt2 krb5kdc[8935](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.206.67@KRBTEST.COM for oryx/127.0.206.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 10:17:47 dist-test-slave-6pt2 krb5kdc[8935](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.206.67@KRBTEST.COM for oryx/127.0.206.126@KRBTEST.COM, Server not found in Kerberos database
I20260619 10:17:47.694983 9959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:47.690498 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:45539 (local address 127.0.206.67:35649)
0619 10:17:47.690683 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0619 10:17:47.690694 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 10:17:47.690754 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 10:17:47.690866 (+ 112us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 10:17:47.690867 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 10:17:47.691015 (+ 148us) client_negotiation.cc:189] Negotiated authn=SASL
0619 10:17:47.691062 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:47.691064 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:47.691661 (+ 597us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:47.691662 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:47.692200 (+ 538us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:47.692202 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:47.692578 (+ 376us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:47.692579 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:47.692642 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:47.693047 (+ 405us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 10:17:47.693051 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 10:17:47.694880 (+ 1829us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.206.126:45539: Server oryx/127.0.206.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":113,"threads_started":1}
W20260619 10:17:47.695025 9957 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 10:17:47.690593 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.67:35649 (local address 127.0.206.126:45539)
0619 10:17:47.690672 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0619 10:17:47.690674 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 10:17:47.690759 (+ 85us) server_negotiation.cc:407] Connection header received
0619 10:17:47.690799 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:17:47.690800 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:17:47.690812 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:17:47.690861 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:17:47.691118 (+ 257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:47.691598 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:47.692290 (+ 692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:47.692493 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:47.692576 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:47.694962 (+ 2386us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.206.67:35649: BlockingRecv error: recv got EOF from 127.0.206.67:35649 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
Jun 19 10:17:48 dist-test-slave-6pt2 krb5kdc[8935](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.206.65@KRBTEST.COM for oryx/127.0.206.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 10:17:48 dist-test-slave-6pt2 krb5kdc[8935](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.206.65@KRBTEST.COM for oryx/127.0.206.126@KRBTEST.COM, Server not found in Kerberos database
I20260619 10:17:48.449693 9960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:48.445306 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:45539 (local address 127.0.206.65:56759)
0619 10:17:48.445438 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0619 10:17:48.445449 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 10:17:48.445497 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 10:17:48.445634 (+ 137us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 10:17:48.445634 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 10:17:48.445797 (+ 163us) client_negotiation.cc:189] Negotiated authn=SASL
0619 10:17:48.445846 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:48.445848 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:48.446463 (+ 615us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:48.446464 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:48.447042 (+ 578us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:48.447045 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:48.447366 (+ 321us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:48.447367 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:48.447406 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:48.447907 (+ 501us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 10:17:48.447912 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 10:17:48.449635 (+ 1723us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.206.126:45539: Server oryx/127.0.206.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":76,"threads_started":1}
W20260619 10:17:48.449802 9961 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 10:17:48.445397 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.65:56759 (local address 127.0.206.126:45539)
0619 10:17:48.445520 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0619 10:17:48.445522 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 10:17:48.445529 (+ 7us) server_negotiation.cc:407] Connection header received
0619 10:17:48.445549 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:17:48.445550 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:17:48.445560 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:17:48.445619 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:17:48.445908 (+ 289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:48.446400 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:48.447116 (+ 716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:48.447302 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:48.447355 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:48.449728 (+ 2373us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.206.65:56759: BlockingRecv error: recv got EOF from 127.0.206.65:56759 (error 108)
Metrics: {"server-negotiator.queue_time_us":97,"thread_start_us":67,"threads_started":1}
Jun 19 10:17:48 dist-test-slave-6pt2 krb5kdc[8935](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.206.66@KRBTEST.COM for oryx/127.0.206.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 10:17:48 dist-test-slave-6pt2 krb5kdc[8935](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.206.66@KRBTEST.COM for oryx/127.0.206.126@KRBTEST.COM, Server not found in Kerberos database
I20260619 10:17:48.585160 9962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:48.580993 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:45539 (local address 127.0.206.66:42831)
0619 10:17:48.581132 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0619 10:17:48.581141 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0619 10:17:48.581201 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 10:17:48.581319 (+ 118us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 10:17:48.581319 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 10:17:48.581470 (+ 151us) client_negotiation.cc:189] Negotiated authn=SASL
0619 10:17:48.581518 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:48.581520 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:48.582130 (+ 610us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:48.582131 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:48.582644 (+ 513us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:48.582645 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:48.582938 (+ 293us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:48.582939 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:48.582973 (+ 34us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:48.583409 (+ 436us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 10:17:48.583414 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 10:17:48.585105 (+ 1691us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.206.126:45539: Server oryx/127.0.206.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":87,"threads_started":1}
W20260619 10:17:48.585294 9961 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 10:17:48.581043 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.66:42831 (local address 127.0.206.126:45539)
0619 10:17:48.581107 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0619 10:17:48.581110 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 10:17:48.581200 (+ 90us) server_negotiation.cc:407] Connection header received
0619 10:17:48.581246 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:17:48.581246 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:17:48.581257 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:17:48.581303 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:17:48.581579 (+ 276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:48.582071 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:48.582711 (+ 640us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:48.582879 (+ 168us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:48.582927 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:48.585214 (+ 2287us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.206.66:42831: BlockingRecv error: recv got EOF from 127.0.206.66:42831 (error 108)
Metrics: {"server-negotiator.queue_time_us":38}
Jun 19 10:17:48 dist-test-slave-6pt2 krb5kdc[8935](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.206.67@KRBTEST.COM for oryx/127.0.206.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 10:17:48 dist-test-slave-6pt2 krb5kdc[8935](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.206.67@KRBTEST.COM for oryx/127.0.206.126@KRBTEST.COM, Server not found in Kerberos database
I20260619 10:17:48.700383 9963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:48.695750 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:45539 (local address 127.0.206.67:43597)
0619 10:17:48.695929 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0619 10:17:48.695939 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 10:17:48.695995 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 10:17:48.696153 (+ 158us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 10:17:48.696154 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 10:17:48.696361 (+ 207us) client_negotiation.cc:189] Negotiated authn=SASL
0619 10:17:48.696436 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:48.696438 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:48.697210 (+ 772us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:48.697210 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:48.697726 (+ 516us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:48.697728 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:48.698030 (+ 302us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:48.698031 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:48.698067 (+ 36us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:48.698556 (+ 489us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 10:17:48.698568 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0619 10:17:48.700325 (+ 1757us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.206.126:45539: Server oryx/127.0.206.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":101,"threads_started":1}
W20260619 10:17:48.700541 9961 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 10:17:48.695843 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.67:43597 (local address 127.0.206.126:45539)
0619 10:17:48.695900 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0619 10:17:48.695902 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 10:17:48.695993 (+ 91us) server_negotiation.cc:407] Connection header received
0619 10:17:48.696038 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:17:48.696039 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:17:48.696051 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:17:48.696145 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:17:48.696533 (+ 388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:48.697119 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:48.697799 (+ 680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:48.697971 (+ 172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:48.698018 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:48.700403 (+ 2385us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.206.67:43597: BlockingRecv error: recv got EOF from 127.0.206.67:43597 (error 108)
Metrics: {"server-negotiator.queue_time_us":31}
Jun 19 10:17:49 dist-test-slave-6pt2 krb5kdc[8935](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.206.65@KRBTEST.COM for oryx/127.0.206.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 10:17:49 dist-test-slave-6pt2 krb5kdc[8935](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.206.65@KRBTEST.COM for oryx/127.0.206.126@KRBTEST.COM, Server not found in Kerberos database
I20260619 10:17:49.454437 9964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:49.450188 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:45539 (local address 127.0.206.65:36915)
0619 10:17:49.450313 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0619 10:17:49.450329 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0619 10:17:49.450375 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 10:17:49.450511 (+ 136us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 10:17:49.450512 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 10:17:49.450673 (+ 161us) client_negotiation.cc:189] Negotiated authn=SASL
0619 10:17:49.450720 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:49.450722 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:49.451284 (+ 562us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:49.451285 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:49.451785 (+ 500us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:49.451786 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:49.452103 (+ 317us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:49.452104 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:49.452142 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:49.452580 (+ 438us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 10:17:49.452584 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 10:17:49.454345 (+ 1761us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.206.126:45539: Server oryx/127.0.206.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":99,"thread_start_us":61,"threads_started":1}
W20260619 10:17:49.454479 9965 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 10:17:49.450278 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.65:36915 (local address 127.0.206.126:45539)
0619 10:17:49.450383 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0619 10:17:49.450385 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 10:17:49.450393 (+ 8us) server_negotiation.cc:407] Connection header received
0619 10:17:49.450414 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:17:49.450414 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:17:49.450440 (+ 26us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:17:49.450494 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:17:49.450779 (+ 285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:49.451224 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:49.451850 (+ 626us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:49.452023 (+ 173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:49.452089 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:49.454413 (+ 2324us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.206.65:36915: BlockingRecv error: recv got EOF from 127.0.206.65:36915 (error 108)
Metrics: {"server-negotiator.queue_time_us":87,"thread_start_us":61,"threads_started":1}
Jun 19 10:17:49 dist-test-slave-6pt2 krb5kdc[8935](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.206.66@KRBTEST.COM for oryx/127.0.206.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 10:17:49 dist-test-slave-6pt2 krb5kdc[8935](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.206.66@KRBTEST.COM for oryx/127.0.206.126@KRBTEST.COM, Server not found in Kerberos database
I20260619 10:17:49.590514 9966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:49.585606 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:45539 (local address 127.0.206.66:55341)
0619 10:17:49.585727 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0619 10:17:49.585768 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0619 10:17:49.585869 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 10:17:49.586015 (+ 146us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 10:17:49.586016 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 10:17:49.586208 (+ 192us) client_negotiation.cc:189] Negotiated authn=SASL
0619 10:17:49.586255 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:49.586257 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:49.587162 (+ 905us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:49.587163 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:49.587664 (+ 501us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:49.587665 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:49.588064 (+ 399us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:49.588065 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:49.588125 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:49.588691 (+ 566us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 10:17:49.588696 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 10:17:49.590445 (+ 1749us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.206.126:45539: Server oryx/127.0.206.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":73,"threads_started":1}
W20260619 10:17:49.590631 9965 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 10:17:49.585687 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.66:55341 (local address 127.0.206.126:45539)
0619 10:17:49.585734 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0619 10:17:49.585736 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 10:17:49.585851 (+ 115us) server_negotiation.cc:407] Connection header received
0619 10:17:49.585924 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:17:49.585925 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:17:49.585938 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:17:49.586008 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:17:49.586349 (+ 341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:49.587085 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:49.587778 (+ 693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:49.587962 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:49.588030 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:49.590545 (+ 2515us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.206.66:55341: BlockingRecv error: recv got EOF from 127.0.206.66:55341 (error 108)
Metrics: {"server-negotiator.queue_time_us":24}
Jun 19 10:17:49 dist-test-slave-6pt2 krb5kdc[8935](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.206.67@KRBTEST.COM for oryx/127.0.206.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 10:17:49 dist-test-slave-6pt2 krb5kdc[8935](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.206.67@KRBTEST.COM for oryx/127.0.206.126@KRBTEST.COM, Server not found in Kerberos database
I20260619 10:17:49.705175 9967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:49.700883 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:45539 (local address 127.0.206.67:37415)
0619 10:17:49.701030 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0619 10:17:49.701047 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0619 10:17:49.701105 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 10:17:49.701211 (+ 106us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 10:17:49.701212 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 10:17:49.701361 (+ 149us) client_negotiation.cc:189] Negotiated authn=SASL
0619 10:17:49.701406 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:49.701407 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:49.701991 (+ 584us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:49.701992 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:49.702525 (+ 533us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:49.702526 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:49.702876 (+ 350us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:49.702877 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:49.702918 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:49.703344 (+ 426us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 10:17:49.703349 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 10:17:49.705117 (+ 1768us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.206.126:45539: Server oryx/127.0.206.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":87,"threads_started":1}
W20260619 10:17:49.705307 9965 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 10:17:49.700933 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.67:37415 (local address 127.0.206.126:45539)
0619 10:17:49.700997 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0619 10:17:49.700999 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 10:17:49.701108 (+ 109us) server_negotiation.cc:407] Connection header received
0619 10:17:49.701146 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:17:49.701147 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:17:49.701158 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:17:49.701208 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:17:49.701456 (+ 248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:49.701929 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:49.702624 (+ 695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:49.702803 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:49.702863 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:49.705225 (+ 2362us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.206.67:37415: BlockingRecv error: recv got EOF from 127.0.206.67:37415 (error 108)
Metrics: {"server-negotiator.queue_time_us":38}
Jun 19 10:17:50 dist-test-slave-6pt2 krb5kdc[8935](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.206.65@KRBTEST.COM for oryx/127.0.206.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 10:17:50 dist-test-slave-6pt2 krb5kdc[8935](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.206.65@KRBTEST.COM for oryx/127.0.206.126@KRBTEST.COM, Server not found in Kerberos database
I20260619 10:17:50.459574 9968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:50.454953 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:45539 (local address 127.0.206.65:43481)
0619 10:17:50.455081 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0619 10:17:50.455091 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 10:17:50.455138 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 10:17:50.455290 (+ 152us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 10:17:50.455290 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 10:17:50.455437 (+ 147us) client_negotiation.cc:189] Negotiated authn=SASL
0619 10:17:50.455485 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:50.455487 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:50.456097 (+ 610us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:50.456098 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:50.456606 (+ 508us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:50.456607 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:50.456980 (+ 373us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:50.456981 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:50.457018 (+ 37us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:50.457450 (+ 432us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 10:17:50.457454 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 10:17:50.459507 (+ 2053us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.206.126:45539: Server oryx/127.0.206.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":76,"threads_started":1}
W20260619 10:17:50.459677 9969 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 10:17:50.455024 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.65:43481 (local address 127.0.206.126:45539)
0619 10:17:50.455164 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0619 10:17:50.455166 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 10:17:50.455174 (+ 8us) server_negotiation.cc:407] Connection header received
0619 10:17:50.455195 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:17:50.455195 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:17:50.455205 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:17:50.455267 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:17:50.455546 (+ 279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:50.456035 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:50.456681 (+ 646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:50.456900 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:50.456972 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:50.459583 (+ 2611us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.206.65:43481: BlockingRecv error: recv got EOF from 127.0.206.65:43481 (error 108)
Metrics: {"server-negotiator.queue_time_us":121,"thread_start_us":88,"threads_started":1}
Jun 19 10:17:50 dist-test-slave-6pt2 krb5kdc[8935](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.206.66@KRBTEST.COM for oryx/127.0.206.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 10:17:50 dist-test-slave-6pt2 krb5kdc[8935](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.206.66@KRBTEST.COM for oryx/127.0.206.126@KRBTEST.COM, Server not found in Kerberos database
I20260619 10:17:50.598829 9970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:50.590956 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:45539 (local address 127.0.206.66:50937)
0619 10:17:50.591123 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0619 10:17:50.591134 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 10:17:50.591193 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 10:17:50.591333 (+ 140us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 10:17:50.591333 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 10:17:50.591504 (+ 171us) client_negotiation.cc:189] Negotiated authn=SASL
0619 10:17:50.591549 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:50.591550 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:50.592193 (+ 643us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:50.592194 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:50.592697 (+ 503us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:50.592699 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:50.593045 (+ 346us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:50.593046 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:50.593101 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:50.596756 (+ 3655us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 10:17:50.596761 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 10:17:50.598737 (+ 1976us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.206.126:45539: Server oryx/127.0.206.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":81,"threads_started":1}
W20260619 10:17:50.598930 9969 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 10:17:50.591041 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.66:50937 (local address 127.0.206.126:45539)
0619 10:17:50.591110 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0619 10:17:50.591112 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 10:17:50.591177 (+ 65us) server_negotiation.cc:407] Connection header received
0619 10:17:50.591243 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:17:50.591244 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:17:50.591255 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:17:50.591316 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:17:50.591625 (+ 309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:50.592112 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:50.592779 (+ 667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:50.592964 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:50.593046 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:50.598850 (+ 5804us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.206.66:50937: BlockingRecv error: recv got EOF from 127.0.206.66:50937 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
Jun 19 10:17:50 dist-test-slave-6pt2 krb5kdc[8935](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.206.67@KRBTEST.COM for oryx/127.0.206.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 10:17:50 dist-test-slave-6pt2 krb5kdc[8935](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.206.67@KRBTEST.COM for oryx/127.0.206.126@KRBTEST.COM, Server not found in Kerberos database
I20260619 10:17:50.710281 9971 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:50.705620 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:45539 (local address 127.0.206.67:44491)
0619 10:17:50.705748 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0619 10:17:50.705758 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 10:17:50.705845 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 10:17:50.705987 (+ 142us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 10:17:50.705989 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 10:17:50.706159 (+ 170us) client_negotiation.cc:189] Negotiated authn=SASL
0619 10:17:50.706204 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:50.706206 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:50.706958 (+ 752us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:50.706959 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:50.707476 (+ 517us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:50.707478 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:50.707831 (+ 353us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:50.707832 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:50.707901 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:50.708375 (+ 474us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 10:17:50.708380 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 10:17:50.710176 (+ 1796us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.206.126:45539: Server oryx/127.0.206.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":77,"threads_started":1}
W20260619 10:17:50.710309 9969 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 10:17:50.705667 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.67:44491 (local address 127.0.206.126:45539)
0619 10:17:50.705714 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0619 10:17:50.705715 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 10:17:50.705818 (+ 103us) server_negotiation.cc:407] Connection header received
0619 10:17:50.705906 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:17:50.705907 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:17:50.705922 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:17:50.705977 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:17:50.706285 (+ 308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:50.706872 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:50.707559 (+ 687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:50.707749 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:50.707830 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:50.710245 (+ 2415us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.206.67:44491: BlockingRecv error: recv got EOF from 127.0.206.67:44491 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
Jun 19 10:17:51 dist-test-slave-6pt2 krb5kdc[8935](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.206.65@KRBTEST.COM for oryx/127.0.206.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 10:17:51 dist-test-slave-6pt2 krb5kdc[8935](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.206.65@KRBTEST.COM for oryx/127.0.206.126@KRBTEST.COM, Server not found in Kerberos database
I20260619 10:17:51.464951 9972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:51.460034 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:45539 (local address 127.0.206.65:58873)
0619 10:17:51.460191 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0619 10:17:51.460206 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0619 10:17:51.460290 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 10:17:51.460465 (+ 175us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 10:17:51.460466 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 10:17:51.460685 (+ 219us) client_negotiation.cc:189] Negotiated authn=SASL
0619 10:17:51.460742 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:51.460743 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:51.461459 (+ 716us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:51.461460 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:51.462173 (+ 713us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:51.462175 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:51.462510 (+ 335us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:51.462511 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:51.462546 (+ 35us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:51.463121 (+ 575us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 10:17:51.463126 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 10:17:51.464874 (+ 1748us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.206.126:45539: Server oryx/127.0.206.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":78,"threads_started":1}
W20260619 10:17:51.465013 9973 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 10:17:51.460069 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.65:58873 (local address 127.0.206.126:45539)
0619 10:17:51.460191 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0619 10:17:51.460193 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 10:17:51.460279 (+ 86us) server_negotiation.cc:407] Connection header received
0619 10:17:51.460353 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:17:51.460353 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:17:51.460362 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:17:51.460455 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:17:51.460807 (+ 352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:51.461368 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:51.462238 (+ 870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:51.462430 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:51.462484 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:51.464961 (+ 2477us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.206.65:58873: BlockingRecv error: recv got EOF from 127.0.206.65:58873 (error 108)
Metrics: {"server-negotiator.queue_time_us":97,"thread_start_us":49,"threads_started":1}
Jun 19 10:17:51 dist-test-slave-6pt2 krb5kdc[8935](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.206.66@KRBTEST.COM for oryx/127.0.206.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 10:17:51 dist-test-slave-6pt2 krb5kdc[8935](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.206.66@KRBTEST.COM for oryx/127.0.206.126@KRBTEST.COM, Server not found in Kerberos database
I20260619 10:17:51.603775 9974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:51.599584 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:45539 (local address 127.0.206.66:45341)
0619 10:17:51.599730 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0619 10:17:51.599738 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0619 10:17:51.599788 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 10:17:51.599910 (+ 122us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 10:17:51.599911 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 10:17:51.600080 (+ 169us) client_negotiation.cc:189] Negotiated authn=SASL
0619 10:17:51.600128 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:51.600129 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:51.600680 (+ 551us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:51.600681 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:51.601147 (+ 466us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:51.601148 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:51.601442 (+ 294us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:51.601443 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:51.601504 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:51.601985 (+ 481us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 10:17:51.601990 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 10:17:51.603675 (+ 1685us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.206.126:45539: Server oryx/127.0.206.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":68,"threads_started":1}
W20260619 10:17:51.603828 9973 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 10:17:51.599645 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.66:45341 (local address 127.0.206.126:45539)
0619 10:17:51.599709 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0619 10:17:51.599711 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 10:17:51.599790 (+ 79us) server_negotiation.cc:407] Connection header received
0619 10:17:51.599830 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:17:51.599830 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:17:51.599840 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:17:51.599892 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:17:51.600179 (+ 287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:51.600620 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:51.601205 (+ 585us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:51.601366 (+ 161us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:51.601412 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:51.603770 (+ 2358us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.206.66:45341: BlockingRecv error: recv got EOF from 127.0.206.66:45341 (error 108)
Metrics: {"server-negotiator.queue_time_us":37}
Jun 19 10:17:51 dist-test-slave-6pt2 krb5kdc[8935](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.206.67@KRBTEST.COM for oryx/127.0.206.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 10:17:51 dist-test-slave-6pt2 krb5kdc[8935](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.206.67@KRBTEST.COM for oryx/127.0.206.126@KRBTEST.COM, Server not found in Kerberos database
I20260619 10:17:51.715262 9975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:51.710812 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:45539 (local address 127.0.206.67:51425)
0619 10:17:51.710990 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0619 10:17:51.711001 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 10:17:51.711047 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 10:17:51.711170 (+ 123us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 10:17:51.711171 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 10:17:51.711315 (+ 144us) client_negotiation.cc:189] Negotiated authn=SASL
0619 10:17:51.711379 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:51.711381 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:51.712014 (+ 633us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:51.712015 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:51.712515 (+ 500us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:51.712516 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:51.712836 (+ 320us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:51.712837 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:51.712892 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:51.713436 (+ 544us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 10:17:51.713441 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 10:17:51.715205 (+ 1764us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.206.126:45539: Server oryx/127.0.206.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":69,"threads_started":1}
W20260619 10:17:51.715353 9973 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 10:17:51.711000 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.67:51425 (local address 127.0.206.126:45539)
0619 10:17:51.711064 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0619 10:17:51.711065 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 10:17:51.711074 (+ 9us) server_negotiation.cc:407] Connection header received
0619 10:17:51.711094 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:17:51.711095 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:17:51.711105 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:17:51.711152 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:17:51.711446 (+ 294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:51.711952 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:51.712575 (+ 623us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:51.712742 (+ 167us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:51.712788 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:51.715287 (+ 2499us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.206.67:51425: BlockingRecv error: recv got EOF from 127.0.206.67:51425 (error 108)
Metrics: {"server-negotiator.queue_time_us":36}
Jun 19 10:17:52 dist-test-slave-6pt2 krb5kdc[8935](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.206.65@KRBTEST.COM for oryx/127.0.206.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 10:17:52 dist-test-slave-6pt2 krb5kdc[8935](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.206.65@KRBTEST.COM for oryx/127.0.206.126@KRBTEST.COM, Server not found in Kerberos database
I20260619 10:17:52.469560 9976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:52.465411 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:45539 (local address 127.0.206.65:56283)
0619 10:17:52.465551 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0619 10:17:52.465562 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 10:17:52.465612 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 10:17:52.465744 (+ 132us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 10:17:52.465745 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 10:17:52.465905 (+ 160us) client_negotiation.cc:189] Negotiated authn=SASL
0619 10:17:52.465952 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:52.465953 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:52.466564 (+ 611us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:52.466564 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:52.467119 (+ 555us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:52.467121 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:52.467492 (+ 371us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:52.467493 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:52.467535 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:52.467936 (+ 401us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 10:17:52.467940 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 10:17:52.469498 (+ 1558us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.206.126:45539: Server oryx/127.0.206.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":84,"threads_started":1}
W20260619 10:17:52.469703 9977 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 10:17:52.465490 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.65:56283 (local address 127.0.206.126:45539)
0619 10:17:52.465622 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0619 10:17:52.465624 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 10:17:52.465633 (+ 9us) server_negotiation.cc:407] Connection header received
0619 10:17:52.465662 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:17:52.465662 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:17:52.465672 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:17:52.465727 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:17:52.466013 (+ 286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:52.466503 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:52.467207 (+ 704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:52.467427 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:52.467483 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:52.469585 (+ 2102us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.206.65:56283: BlockingRecv error: recv got EOF from 127.0.206.65:56283 (error 108)
Metrics: {"server-negotiator.queue_time_us":113,"thread_start_us":86,"threads_started":1}
Jun 19 10:17:52 dist-test-slave-6pt2 krb5kdc[8935](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.206.66@KRBTEST.COM for oryx/127.0.206.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 10:17:52 dist-test-slave-6pt2 krb5kdc[8935](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.206.66@KRBTEST.COM for oryx/127.0.206.126@KRBTEST.COM, Server not found in Kerberos database
I20260619 10:17:52.609486 9978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:52.604173 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:45539 (local address 127.0.206.66:54129)
0619 10:17:52.604281 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0619 10:17:52.604293 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 10:17:52.604349 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 10:17:52.604455 (+ 106us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 10:17:52.604455 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 10:17:52.604611 (+ 156us) client_negotiation.cc:189] Negotiated authn=SASL
0619 10:17:52.604656 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:52.604657 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:52.605751 (+ 1094us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:52.605752 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:52.606290 (+ 538us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:52.606292 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:52.606766 (+ 474us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:52.606767 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:52.606811 (+ 44us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:52.607304 (+ 493us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 10:17:52.607310 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0619 10:17:52.609373 (+ 2063us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.206.126:45539: Server oryx/127.0.206.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":88,"thread_start_us":62,"threads_started":1}
W20260619 10:17:52.609809 9977 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 10:17:52.604236 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.66:54129 (local address 127.0.206.126:45539)
0619 10:17:52.604278 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0619 10:17:52.604280 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 10:17:52.604351 (+ 71us) server_negotiation.cc:407] Connection header received
0619 10:17:52.604388 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:17:52.604389 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:17:52.604399 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:17:52.604448 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:17:52.604717 (+ 269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:52.605648 (+ 931us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:52.606381 (+ 733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:52.606648 (+ 267us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:52.606706 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:52.609478 (+ 2772us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.206.66:54129: BlockingRecv error: recv got EOF from 127.0.206.66:54129 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
Jun 19 10:17:52 dist-test-slave-6pt2 krb5kdc[8935](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.206.67@KRBTEST.COM for oryx/127.0.206.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 10:17:52 dist-test-slave-6pt2 krb5kdc[8935](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.206.67@KRBTEST.COM for oryx/127.0.206.126@KRBTEST.COM, Server not found in Kerberos database
I20260619 10:17:52.719872 9979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:52.715710 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:45539 (local address 127.0.206.67:45635)
0619 10:17:52.715848 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0619 10:17:52.715858 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 10:17:52.715910 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 10:17:52.716031 (+ 121us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 10:17:52.716032 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 10:17:52.716189 (+ 157us) client_negotiation.cc:189] Negotiated authn=SASL
0619 10:17:52.716234 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:52.716236 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:52.716833 (+ 597us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:52.716834 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:52.717326 (+ 492us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:52.717327 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:52.717609 (+ 282us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:52.717610 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:52.717646 (+ 36us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:52.718121 (+ 475us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 10:17:52.718125 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 10:17:52.719807 (+ 1682us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.206.126:45539: Server oryx/127.0.206.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":87,"threads_started":1}
W20260619 10:17:52.719935 9977 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 10:17:52.715777 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.67:45635 (local address 127.0.206.126:45539)
0619 10:17:52.715839 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0619 10:17:52.715841 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 10:17:52.715896 (+ 55us) server_negotiation.cc:407] Connection header received
0619 10:17:52.715951 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:17:52.715951 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:17:52.715964 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:17:52.716015 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:17:52.716292 (+ 277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:52.716771 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:52.717387 (+ 616us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:52.717552 (+ 165us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:52.717598 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:52.719889 (+ 2291us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.206.67:45635: BlockingRecv error: recv got EOF from 127.0.206.67:45635 (error 108)
Metrics: {"server-negotiator.queue_time_us":34}
Jun 19 10:17:53 dist-test-slave-6pt2 krb5kdc[8935](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.206.65@KRBTEST.COM for oryx/127.0.206.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 10:17:53 dist-test-slave-6pt2 krb5kdc[8935](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.206.65@KRBTEST.COM for oryx/127.0.206.126@KRBTEST.COM, Server not found in Kerberos database
I20260619 10:17:53.474370 9980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:53.470097 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:45539 (local address 127.0.206.65:50673)
0619 10:17:53.470239 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0619 10:17:53.470260 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0619 10:17:53.470324 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 10:17:53.470456 (+ 132us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 10:17:53.470456 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 10:17:53.470680 (+ 224us) client_negotiation.cc:189] Negotiated authn=SASL
0619 10:17:53.470727 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:53.470729 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:53.471358 (+ 629us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:53.471358 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:53.471877 (+ 519us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:53.471879 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:53.472191 (+ 312us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:53.472191 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:53.472227 (+ 36us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:53.472701 (+ 474us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 10:17:53.472705 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 10:17:53.474306 (+ 1601us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.206.126:45539: Server oryx/127.0.206.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":65,"threads_started":1}
W20260619 10:17:53.474479 9981 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 10:17:53.470142 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.65:50673 (local address 127.0.206.126:45539)
0619 10:17:53.470281 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0619 10:17:53.470282 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 10:17:53.470307 (+ 25us) server_negotiation.cc:407] Connection header received
0619 10:17:53.470379 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:17:53.470379 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:17:53.470390 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:17:53.470441 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:17:53.470796 (+ 355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:53.471289 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:53.471956 (+ 667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:53.472133 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:53.472179 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:53.474383 (+ 2204us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.206.65:50673: BlockingRecv error: recv got EOF from 127.0.206.65:50673 (error 108)
Metrics: {"server-negotiator.queue_time_us":116,"thread_start_us":77,"threads_started":1}
Jun 19 10:17:53 dist-test-slave-6pt2 krb5kdc[8935](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.206.66@KRBTEST.COM for oryx/127.0.206.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 10:17:53 dist-test-slave-6pt2 krb5kdc[8935](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.206.66@KRBTEST.COM for oryx/127.0.206.126@KRBTEST.COM, Server not found in Kerberos database
I20260619 10:17:53.614235 9982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:53.610055 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:45539 (local address 127.0.206.66:60935)
0619 10:17:53.610213 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0619 10:17:53.610223 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 10:17:53.610281 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 10:17:53.610425 (+ 144us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 10:17:53.610426 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 10:17:53.610583 (+ 157us) client_negotiation.cc:189] Negotiated authn=SASL
0619 10:17:53.610655 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:53.610656 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:53.611266 (+ 610us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:53.611266 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:53.611753 (+ 487us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:53.611754 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:53.612064 (+ 310us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:53.612065 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:53.612101 (+ 36us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:53.612518 (+ 417us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 10:17:53.612523 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 10:17:53.614178 (+ 1655us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.206.126:45539: Server oryx/127.0.206.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":106,"threads_started":1}
W20260619 10:17:53.614332 9981 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 10:17:53.610209 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.66:60935 (local address 127.0.206.126:45539)
0619 10:17:53.610297 (+ 88us) server_negotiation.cc:206] Beginning negotiation
0619 10:17:53.610299 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 10:17:53.610312 (+ 13us) server_negotiation.cc:407] Connection header received
0619 10:17:53.610342 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:17:53.610342 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:17:53.610352 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:17:53.610406 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:17:53.610716 (+ 310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:53.611195 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:53.611819 (+ 624us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:53.611987 (+ 168us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:53.612033 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:53.614265 (+ 2232us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.206.66:60935: BlockingRecv error: recv got EOF from 127.0.206.66:60935 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
Jun 19 10:17:53 dist-test-slave-6pt2 krb5kdc[8935](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.206.67@KRBTEST.COM for oryx/127.0.206.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 10:17:53 dist-test-slave-6pt2 krb5kdc[8935](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.206.67@KRBTEST.COM for oryx/127.0.206.126@KRBTEST.COM, Server not found in Kerberos database
I20260619 10:17:53.725106 9983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:53.720321 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:45539 (local address 127.0.206.67:39649)
0619 10:17:53.720468 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0619 10:17:53.720478 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 10:17:53.720545 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 10:17:53.720683 (+ 138us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 10:17:53.720684 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 10:17:53.720854 (+ 170us) client_negotiation.cc:189] Negotiated authn=SASL
0619 10:17:53.720898 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:53.720900 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:53.721531 (+ 631us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:53.721532 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:53.722099 (+ 567us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:53.722101 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:53.722437 (+ 336us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:53.722438 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:53.722518 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:53.723085 (+ 567us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 10:17:53.723090 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 10:17:53.725042 (+ 1952us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.206.126:45539: Server oryx/127.0.206.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":85,"threads_started":1}
W20260619 10:17:53.725215 9981 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 10:17:53.720480 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.67:39649 (local address 127.0.206.126:45539)
0619 10:17:53.720561 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0619 10:17:53.720563 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 10:17:53.720571 (+ 8us) server_negotiation.cc:407] Connection header received
0619 10:17:53.720596 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:17:53.720597 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:17:53.720616 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:17:53.720676 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:17:53.720976 (+ 300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:53.721453 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:53.722181 (+ 728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:53.722367 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:53.722439 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:53.725126 (+ 2687us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.206.67:39649: BlockingRecv error: recv got EOF from 127.0.206.67:39649 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
Jun 19 10:17:54 dist-test-slave-6pt2 krb5kdc[8935](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.206.65@KRBTEST.COM for oryx/127.0.206.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 10:17:54 dist-test-slave-6pt2 krb5kdc[8935](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.206.65@KRBTEST.COM for oryx/127.0.206.126@KRBTEST.COM, Server not found in Kerberos database
I20260619 10:17:54.479321 9984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:54.474795 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:45539 (local address 127.0.206.65:33417)
0619 10:17:54.474944 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0619 10:17:54.474956 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 10:17:54.475011 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 10:17:54.475136 (+ 125us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 10:17:54.475137 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 10:17:54.475299 (+ 162us) client_negotiation.cc:189] Negotiated authn=SASL
0619 10:17:54.475344 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:54.475346 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:54.476050 (+ 704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:54.476050 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:54.476567 (+ 517us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:54.476568 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:54.476971 (+ 403us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:54.476972 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:54.477010 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:54.477491 (+ 481us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 10:17:54.477495 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 10:17:54.479265 (+ 1770us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.206.126:45539: Server oryx/127.0.206.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":104,"threads_started":1}
W20260619 10:17:54.479422 9985 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 10:17:54.474858 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.65:33417 (local address 127.0.206.126:45539)
0619 10:17:54.474988 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0619 10:17:54.474990 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 10:17:54.474999 (+ 9us) server_negotiation.cc:407] Connection header received
0619 10:17:54.475054 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:17:54.475054 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:17:54.475063 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:17:54.475121 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:17:54.475406 (+ 285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:54.475979 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:54.476637 (+ 658us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:54.476912 (+ 275us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:54.476961 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:54.479348 (+ 2387us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.206.65:33417: BlockingRecv error: recv got EOF from 127.0.206.65:33417 (error 108)
Metrics: {"server-negotiator.queue_time_us":107,"thread_start_us":62,"threads_started":1}
Jun 19 10:17:54 dist-test-slave-6pt2 krb5kdc[8935](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.206.66@KRBTEST.COM for oryx/127.0.206.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 10:17:54 dist-test-slave-6pt2 krb5kdc[8935](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.206.66@KRBTEST.COM for oryx/127.0.206.126@KRBTEST.COM, Server not found in Kerberos database
I20260619 10:17:54.618990 9986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:54.614698 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:45539 (local address 127.0.206.66:55123)
0619 10:17:54.614834 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0619 10:17:54.614852 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0619 10:17:54.614918 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 10:17:54.615038 (+ 120us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 10:17:54.615039 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 10:17:54.615196 (+ 157us) client_negotiation.cc:189] Negotiated authn=SASL
0619 10:17:54.615242 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:54.615244 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:54.615844 (+ 600us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:54.615845 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:54.616357 (+ 512us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:54.616359 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:54.616658 (+ 299us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:54.616658 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:54.616694 (+ 36us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:54.617110 (+ 416us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 10:17:54.617114 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 10:17:54.618895 (+ 1781us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.206.126:45539: Server oryx/127.0.206.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":81,"threads_started":1}
W20260619 10:17:54.619030 9985 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 10:17:54.614790 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.66:55123 (local address 127.0.206.126:45539)
0619 10:17:54.614854 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0619 10:17:54.614856 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 10:17:54.614919 (+ 63us) server_negotiation.cc:407] Connection header received
0619 10:17:54.614963 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:17:54.614963 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:17:54.614973 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:17:54.615022 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:17:54.615296 (+ 274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:54.615778 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:54.616421 (+ 643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:54.616598 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:54.616646 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:54.618969 (+ 2323us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.206.66:55123: BlockingRecv error: recv got EOF from 127.0.206.66:55123 (error 108)
Metrics: {"server-negotiator.queue_time_us":30}
Jun 19 10:17:54 dist-test-slave-6pt2 krb5kdc[8935](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.206.67@KRBTEST.COM for oryx/127.0.206.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 10:17:54 dist-test-slave-6pt2 krb5kdc[8935](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.206.67@KRBTEST.COM for oryx/127.0.206.126@KRBTEST.COM, Server not found in Kerberos database
I20260619 10:17:54.730293 9987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:54.725635 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:45539 (local address 127.0.206.67:43959)
0619 10:17:54.725777 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0619 10:17:54.725792 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0619 10:17:54.725882 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 10:17:54.726056 (+ 174us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 10:17:54.726057 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 10:17:54.726243 (+ 186us) client_negotiation.cc:189] Negotiated authn=SASL
0619 10:17:54.726286 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:54.726288 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:54.727016 (+ 728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:54.727017 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:54.727543 (+ 526us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:54.727544 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:54.727925 (+ 381us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:54.727927 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:54.727987 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:54.728449 (+ 462us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 10:17:54.728454 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 10:17:54.730238 (+ 1784us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.206.126:45539: Server oryx/127.0.206.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":55,"threads_started":1}
W20260619 10:17:54.730415 9985 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 10:17:54.725793 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.67:43959 (local address 127.0.206.126:45539)
0619 10:17:54.725851 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0619 10:17:54.725853 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 10:17:54.725862 (+ 9us) server_negotiation.cc:407] Connection header received
0619 10:17:54.725966 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:17:54.725967 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:17:54.725988 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:17:54.726051 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:17:54.726364 (+ 313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:54.726932 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:54.727627 (+ 695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:54.727824 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:54.727912 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:54.730320 (+ 2408us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.206.67:43959: BlockingRecv error: recv got EOF from 127.0.206.67:43959 (error 108)
Metrics: {"server-negotiator.queue_time_us":35}
Jun 19 10:17:55 dist-test-slave-6pt2 krb5kdc[8935](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.206.65@KRBTEST.COM for oryx/127.0.206.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 10:17:55 dist-test-slave-6pt2 krb5kdc[8935](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.206.65@KRBTEST.COM for oryx/127.0.206.126@KRBTEST.COM, Server not found in Kerberos database
I20260619 10:17:55.484050 9988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:55.479735 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:45539 (local address 127.0.206.65:54043)
0619 10:17:55.479871 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0619 10:17:55.479881 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 10:17:55.479931 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 10:17:55.480061 (+ 130us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 10:17:55.480062 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 10:17:55.480224 (+ 162us) client_negotiation.cc:189] Negotiated authn=SASL
0619 10:17:55.480271 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:55.480272 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:55.480889 (+ 617us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:55.480889 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:55.481403 (+ 514us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:55.481405 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:55.481715 (+ 310us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:55.481716 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:55.481753 (+ 37us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:55.482191 (+ 438us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 10:17:55.482195 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 10:17:55.483931 (+ 1736us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.206.126:45539: Server oryx/127.0.206.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":81,"threads_started":1}
W20260619 10:17:55.484108 9989 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 10:17:55.479811 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.65:54043 (local address 127.0.206.126:45539)
0619 10:17:55.479922 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0619 10:17:55.479923 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 10:17:55.479931 (+ 8us) server_negotiation.cc:407] Connection header received
0619 10:17:55.479987 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:17:55.479988 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:17:55.479997 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:17:55.480043 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:17:55.480330 (+ 287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:55.480829 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:55.481467 (+ 638us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:55.481655 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:55.481702 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:55.484039 (+ 2337us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.206.65:54043: BlockingRecv error: recv got EOF from 127.0.206.65:54043 (error 108)
Metrics: {"server-negotiator.queue_time_us":94,"thread_start_us":45,"threads_started":1}
Jun 19 10:17:55 dist-test-slave-6pt2 krb5kdc[8935](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.206.66@KRBTEST.COM for oryx/127.0.206.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 10:17:55 dist-test-slave-6pt2 krb5kdc[8935](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.206.66@KRBTEST.COM for oryx/127.0.206.126@KRBTEST.COM, Server not found in Kerberos database
I20260619 10:17:55.627197 9990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:55.619536 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:45539 (local address 127.0.206.66:40269)
0619 10:17:55.619661 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0619 10:17:55.619695 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0619 10:17:55.619763 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 10:17:55.619916 (+ 153us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 10:17:55.619917 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 10:17:55.620089 (+ 172us) client_negotiation.cc:189] Negotiated authn=SASL
0619 10:17:55.620132 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:55.620133 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:55.620772 (+ 639us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:55.620773 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:55.621299 (+ 526us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:55.621300 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:55.621672 (+ 372us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:55.621672 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:55.621734 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:55.625240 (+ 3506us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 10:17:55.625245 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 10:17:55.627088 (+ 1843us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.206.126:45539: Server oryx/127.0.206.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":76,"threads_started":1}
W20260619 10:17:55.627233 9989 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 10:17:55.619652 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.66:40269 (local address 127.0.206.126:45539)
0619 10:17:55.619728 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0619 10:17:55.619730 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 10:17:55.619743 (+ 13us) server_negotiation.cc:407] Connection header received
0619 10:17:55.619851 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:17:55.619852 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:17:55.619863 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:17:55.619909 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:17:55.620211 (+ 302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:55.620691 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:55.621383 (+ 692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:55.621588 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:55.621670 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:55.627169 (+ 5499us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.206.66:40269: BlockingRecv error: recv got EOF from 127.0.206.66:40269 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
Jun 19 10:17:55 dist-test-slave-6pt2 krb5kdc[8935](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.206.67@KRBTEST.COM for oryx/127.0.206.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 10:17:55 dist-test-slave-6pt2 krb5kdc[8935](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.206.67@KRBTEST.COM for oryx/127.0.206.126@KRBTEST.COM, Server not found in Kerberos database
I20260619 10:17:55.735529 9991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:55.730713 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:45539 (local address 127.0.206.67:52189)
0619 10:17:55.730880 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0619 10:17:55.730892 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 10:17:55.730980 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 10:17:55.731149 (+ 169us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 10:17:55.731150 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 10:17:55.731324 (+ 174us) client_negotiation.cc:189] Negotiated authn=SASL
0619 10:17:55.731367 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:55.731369 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:55.732018 (+ 649us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:55.732019 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:55.732558 (+ 539us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:55.732560 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:55.732920 (+ 360us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:55.732921 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:55.732978 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:55.733520 (+ 542us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 10:17:55.733526 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0619 10:17:55.735470 (+ 1944us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.206.126:45539: Server oryx/127.0.206.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":77,"threads_started":1}
W20260619 10:17:55.735656 9989 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 10:17:55.730800 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.67:52189 (local address 127.0.206.126:45539)
0619 10:17:55.730867 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0619 10:17:55.730869 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 10:17:55.730961 (+ 92us) server_negotiation.cc:407] Connection header received
0619 10:17:55.731048 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:17:55.731049 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:17:55.731063 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:17:55.731131 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:17:55.731442 (+ 311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:55.731936 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:55.732640 (+ 704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:55.732836 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:55.732919 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:55.735559 (+ 2640us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.206.67:52189: BlockingRecv error: recv got EOF from 127.0.206.67:52189 (error 108)
Metrics: {"server-negotiator.queue_time_us":38}
Jun 19 10:17:56 dist-test-slave-6pt2 krb5kdc[8935](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.206.65@KRBTEST.COM for oryx/127.0.206.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 10:17:56 dist-test-slave-6pt2 krb5kdc[8935](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.206.65@KRBTEST.COM for oryx/127.0.206.126@KRBTEST.COM, Server not found in Kerberos database
I20260619 10:17:56.489652 9992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:56.484649 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:45539 (local address 127.0.206.65:42069)
0619 10:17:56.484781 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0619 10:17:56.484792 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 10:17:56.484872 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 10:17:56.485034 (+ 162us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 10:17:56.485035 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 10:17:56.485193 (+ 158us) client_negotiation.cc:189] Negotiated authn=SASL
0619 10:17:56.485240 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:56.485241 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:56.485843 (+ 602us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:56.485843 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:56.486534 (+ 691us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:56.486537 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:56.487136 (+ 599us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:56.487137 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:56.487259 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:56.487826 (+ 567us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 10:17:56.487834 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 10:17:56.489542 (+ 1708us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.206.126:45539: Server oryx/127.0.206.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":80,"threads_started":1}
W20260619 10:17:56.489683 9993 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 10:17:56.484689 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.65:42069 (local address 127.0.206.126:45539)
0619 10:17:56.484805 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0619 10:17:56.484808 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 10:17:56.484853 (+ 45us) server_negotiation.cc:407] Connection header received
0619 10:17:56.484939 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:17:56.484940 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:17:56.484953 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:17:56.485036 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:17:56.485302 (+ 266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:56.485777 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:56.486660 (+ 883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:56.487032 (+ 372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:56.487112 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:56.489616 (+ 2504us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.206.65:42069: BlockingRecv error: recv got EOF from 127.0.206.65:42069 (error 108)
Metrics: {"server-negotiator.queue_time_us":96,"thread_start_us":68,"threads_started":1}
Jun 19 10:17:56 dist-test-slave-6pt2 krb5kdc[8935](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.206.66@KRBTEST.COM for oryx/127.0.206.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 10:17:56 dist-test-slave-6pt2 krb5kdc[8935](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.0.206.66@KRBTEST.COM for oryx/127.0.206.126@KRBTEST.COM, Server not found in Kerberos database
I20260619 10:17:56.631958 9994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:56.627682 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:45539 (local address 127.0.206.66:50529)
0619 10:17:56.627808 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0619 10:17:56.627818 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 10:17:56.627884 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 10:17:56.627997 (+ 113us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 10:17:56.627998 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 10:17:56.628146 (+ 148us) client_negotiation.cc:189] Negotiated authn=SASL
0619 10:17:56.628202 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:56.628204 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:56.628788 (+ 584us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:56.628789 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:56.629284 (+ 495us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:56.629285 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:56.629668 (+ 383us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:56.629669 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:56.629711 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:56.630117 (+ 406us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 10:17:56.630121 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 10:17:56.631900 (+ 1779us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.0.206.126:45539: Server oryx/127.0.206.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":61,"threads_started":1}
W20260619 10:17:56.632074 9993 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 10:17:56.627741 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.66:50529 (local address 127.0.206.126:45539)
0619 10:17:56.627796 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0619 10:17:56.627797 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 10:17:56.627869 (+ 72us) server_negotiation.cc:407] Connection header received
0619 10:17:56.627925 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:17:56.627926 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:17:56.627935 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:17:56.627982 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:17:56.628258 (+ 276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:56.628727 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:56.629347 (+ 620us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:56.629615 (+ 268us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:56.629665 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:56.631983 (+ 2318us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.206.66:50529: BlockingRecv error: recv got EOF from 127.0.206.66:50529 (error 108)
Metrics: {"server-negotiator.queue_time_us":31}
I20260619 10:17:56.655977 825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTU9OP6/build/release/bin/kudu with pid 9519
I20260619 10:17:56.660597 825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTU9OP6/build/release/bin/kudu with pid 9671
I20260619 10:17:56.664618 825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTU9OP6/build/release/bin/kudu with pid 9803
I20260619 10:17:56.668644 825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTU9OP6/build/release/bin/kudu with pid 9448
2026-06-19T10:17:56Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (17330 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-06-19T10:17:56Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T10:17:56Z Disabled control of system clock
I20260619 10:17:56.695222 825 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTU9OP6/build/release/bin/kudu
/tmp/dist-test-taskTU9OP6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781864224577058-825-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781864224577058-825-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781864224577058-825-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781864224577058-825-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.206.126:44555
--webserver_interface=127.0.206.126
--webserver_port=0
--builtin_ntp_servers=127.0.206.84:46029
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.206.126:44555
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260619 10:17:56.773659 9998 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 10:17:56.773841 9998 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 10:17:56.773861 9998 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 10:17:56.774986 9998 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 10:17:56.775024 9998 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 10:17:56.775038 9998 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 10:17:56.775050 9998 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 10:17:56.775067 9998 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 10:17:56.776296 9998 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.206.84:46029
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781864224577058-825-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781864224577058-825-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.206.126:44555
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.206.126:44555
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781864224577058-825-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.206.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9998
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781864224577058-825-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 09:32:44 UTC on ccycloud.quasar-gbtldp.root.comops.site
build id 4455440
I20260619 10:17:56.776472 9998 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 10:17:56.776685 9998 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 10:17:56.781533 10006 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 10:17:56.781500 9998 server_base.cc:1034] running on GCE node
W20260619 10:17:56.781491 10003 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 10:17:56.781479 10004 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 10:17:56.782002 9998 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 10:17:56.782291 9998 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 10:17:56.783448 9998 hybrid_clock.cc:648] HybridClock initialized: now 1781864276783438 us; error 39 us; skew 500 ppm
I20260619 10:17:56.784646 9998 webserver.cc:460] Webserver started at http://127.0.206.126:45709/ using document root <none> and password file <none>
I20260619 10:17:56.784916 9998 fs_manager.cc:359] Metadata directory not provided
I20260619 10:17:56.784984 9998 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 10:17:56.785157 9998 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 10:17:56.786131 9998 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781864224577058-825-0/minicluster-data/master-0/data/instance:
uuid: "9da1aa50fd2747369372872e6c74d41b"
format_stamp: "Formatted at 2026-06-19 10:17:56 on dist-test-slave-6pt2"
server_key: "e85be5e2c71ae318b19bcdb07057a78d"
server_key_iv: "6aec8a55614ba122f0c564c5c3756691"
server_key_version: "encryptionkey@0"
I20260619 10:17:56.786471 9998 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781864224577058-825-0/minicluster-data/master-0/wal/instance:
uuid: "9da1aa50fd2747369372872e6c74d41b"
format_stamp: "Formatted at 2026-06-19 10:17:56 on dist-test-slave-6pt2"
server_key: "e85be5e2c71ae318b19bcdb07057a78d"
server_key_iv: "6aec8a55614ba122f0c564c5c3756691"
server_key_version: "encryptionkey@0"
I20260619 10:17:56.787703 9998 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260619 10:17:56.788403 10012 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 10:17:56.788559 9998 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260619 10:17:56.788656 9998 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781864224577058-825-0/minicluster-data/master-0/data,/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781864224577058-825-0/minicluster-data/master-0/wal
uuid: "9da1aa50fd2747369372872e6c74d41b"
format_stamp: "Formatted at 2026-06-19 10:17:56 on dist-test-slave-6pt2"
server_key: "e85be5e2c71ae318b19bcdb07057a78d"
server_key_iv: "6aec8a55614ba122f0c564c5c3756691"
server_key_version: "encryptionkey@0"
I20260619 10:17:56.788731 9998 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781864224577058-825-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781864224577058-825-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781864224577058-825-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 10:17:56.798784 9998 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 10:17:56.799031 9998 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 10:17:56.799178 9998 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 10:17:56.803045 9998 rpc_server.cc:307] RPC server started. Bound to: 127.0.206.126:44555
I20260619 10:17:56.803076 10064 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.206.126:44555 every 8 connection(s)
I20260619 10:17:56.803380 9998 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781864224577058-825-0/minicluster-data/master-0/data/info.pb
I20260619 10:17:56.803956 10065 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 10:17:56.806288 10065 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9da1aa50fd2747369372872e6c74d41b: Bootstrap starting.
I20260619 10:17:56.806859 10065 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9da1aa50fd2747369372872e6c74d41b: Neither blocks nor log segments found. Creating new log.
I20260619 10:17:56.807086 10065 log.cc:824] T 00000000000000000000000000000000 P 9da1aa50fd2747369372872e6c74d41b: Log is configured to *not* fsync() on all Append() calls
I20260619 10:17:56.807642 10065 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9da1aa50fd2747369372872e6c74d41b: No bootstrap required, opened a new log
I20260619 10:17:56.808945 825 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTU9OP6/build/release/bin/kudu as pid 9998
I20260619 10:17:56.809043 825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781864224577058-825-0/minicluster-data/master-0/wal/instance
I20260619 10:17:56.809216 825 external_mini_cluster.cc:1428] Setting key c271cfc8ed30c9329bb1e79a5a7d8da7
I20260619 10:17:56.809161 10065 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9da1aa50fd2747369372872e6c74d41b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9da1aa50fd2747369372872e6c74d41b" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 44555 } }
I20260619 10:17:56.809305 10065 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9da1aa50fd2747369372872e6c74d41b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 10:17:56.809345 10065 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9da1aa50fd2747369372872e6c74d41b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9da1aa50fd2747369372872e6c74d41b, State: Initialized, Role: FOLLOWER
I20260619 10:17:56.809471 10065 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9da1aa50fd2747369372872e6c74d41b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9da1aa50fd2747369372872e6c74d41b" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 44555 } }
I20260619 10:17:56.809607 10065 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9da1aa50fd2747369372872e6c74d41b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 10:17:56.809671 10065 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9da1aa50fd2747369372872e6c74d41b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 10:17:56.809758 10065 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9da1aa50fd2747369372872e6c74d41b [term 0 FOLLOWER]: Advancing to term 1
I20260619 10:17:56.810364 10065 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9da1aa50fd2747369372872e6c74d41b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9da1aa50fd2747369372872e6c74d41b" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 44555 } }
I20260619 10:17:56.810467 10065 leader_election.cc:302] T 00000000000000000000000000000000 P 9da1aa50fd2747369372872e6c74d41b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9da1aa50fd2747369372872e6c74d41b; no voters:
I20260619 10:17:56.810667 10065 leader_election.cc:288] T 00000000000000000000000000000000 P 9da1aa50fd2747369372872e6c74d41b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 10:17:56.810688 10070 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9da1aa50fd2747369372872e6c74d41b [term 1 FOLLOWER]: Leader election won for term 1
I20260619 10:17:56.810786 10070 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9da1aa50fd2747369372872e6c74d41b [term 1 LEADER]: Becoming Leader. State: Replica: 9da1aa50fd2747369372872e6c74d41b, State: Running, Role: LEADER
I20260619 10:17:56.810859 10070 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9da1aa50fd2747369372872e6c74d41b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9da1aa50fd2747369372872e6c74d41b" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 44555 } }
I20260619 10:17:56.810907 10065 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9da1aa50fd2747369372872e6c74d41b [sys.catalog]: configured and running, proceeding with master startup.
I20260619 10:17:56.811143 10070 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9da1aa50fd2747369372872e6c74d41b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9da1aa50fd2747369372872e6c74d41b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9da1aa50fd2747369372872e6c74d41b" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 44555 } } }
I20260619 10:17:56.811319 10070 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9da1aa50fd2747369372872e6c74d41b [sys.catalog]: This master's current role is: LEADER
I20260619 10:17:56.811496 10070 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9da1aa50fd2747369372872e6c74d41b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9da1aa50fd2747369372872e6c74d41b. Latest consensus state: current_term: 1 leader_uuid: "9da1aa50fd2747369372872e6c74d41b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9da1aa50fd2747369372872e6c74d41b" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 44555 } } }
I20260619 10:17:56.811574 10070 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9da1aa50fd2747369372872e6c74d41b [sys.catalog]: This master's current role is: LEADER
I20260619 10:17:56.811968 10076 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 10:17:56.812531 10076 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 10:17:56.814158 10076 catalog_manager.cc:1347] Generated new cluster ID: 5df31d930d0044db8a665dd7cc7cca78
I20260619 10:17:56.814236 10076 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 10:17:56.815356 10069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:56.809653 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49820 (local address 127.0.206.126:44555)
0619 10:17:56.809868 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0619 10:17:56.809876 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 10:17:56.809909 (+ 33us) server_negotiation.cc:407] Connection header received
0619 10:17:56.809996 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:17:56.810000 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:17:56.810057 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:17:56.810157 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:17:56.810392 (+ 235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:56.812123 (+ 1731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:56.813238 (+ 1115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:56.813830 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:56.813969 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:56.814658 (+ 689us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 10:17:56.814662 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 10:17:56.814663 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 10:17:56.814822 (+ 159us) server_negotiation.cc:922] calling sasl_server_start()
0619 10:17:56.814851 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 10:17:56.814861 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 10:17:56.814974 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0619 10:17:56.815070 (+ 96us) server_negotiation.cc:299] Negotiation successful
0619 10:17:56.815144 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":140,"thread_start_us":101,"threads_started":1}
I20260619 10:17:56.831396 10076 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 10:17:56.832028 10076 catalog_manager.cc:1504] Loading token signing keys...
I20260619 10:17:56.839135 10076 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9da1aa50fd2747369372872e6c74d41b: Generated new TSK 0
I20260619 10:17:56.839289 10076 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 10:17:56.848222 825 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTU9OP6/build/release/bin/kudu
/tmp/dist-test-taskTU9OP6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781864224577058-825-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781864224577058-825-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781864224577058-825-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781864224577058-825-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.206.65:0
--local_ip_for_outbound_sockets=127.0.206.65
--webserver_interface=127.0.206.65
--webserver_port=0
--tserver_master_addrs=127.0.206.126:44555
--builtin_ntp_servers=127.0.206.84:46029
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260619 10:17:56.932344 10089 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 10:17:56.932545 10089 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 10:17:56.932565 10089 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 10:17:56.933773 10089 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 10:17:56.933815 10089 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 10:17:56.933838 10089 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.206.65
I20260619 10:17:56.935232 10089 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.206.84:46029
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781864224577058-825-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781864224577058-825-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.206.65:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781864224577058-825-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.206.65
--webserver_port=0
--tserver_master_addrs=127.0.206.126:44555
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10089
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.206.65
--log_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781864224577058-825-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 09:32:44 UTC on ccycloud.quasar-gbtldp.root.comops.site
build id 4455440
I20260619 10:17:56.935474 10089 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 10:17:56.935772 10089 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 10:17:56.941565 10097 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 10:17:56.941787 10089 server_base.cc:1034] running on GCE node
W20260619 10:17:56.941825 10094 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 10:17:56.941846 10095 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 10:17:56.942175 10089 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 10:17:56.942404 10089 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 10:17:56.943594 10089 hybrid_clock.cc:648] HybridClock initialized: now 1781864276943577 us; error 32 us; skew 500 ppm
I20260619 10:17:56.944792 10089 webserver.cc:460] Webserver started at http://127.0.206.65:46881/ using document root <none> and password file <none>
I20260619 10:17:56.945081 10089 fs_manager.cc:359] Metadata directory not provided
I20260619 10:17:56.945151 10089 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 10:17:56.945284 10089 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 10:17:56.946308 10089 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781864224577058-825-0/minicluster-data/ts-0/data/instance:
uuid: "2d880769cbd8414491ca90bdd6dbcc05"
format_stamp: "Formatted at 2026-06-19 10:17:56 on dist-test-slave-6pt2"
server_key: "892a6d607d7b1a0b9892fc8f93c41155"
server_key_iv: "1f4048c29724653fd7a0d04d4cc5b441"
server_key_version: "encryptionkey@0"
I20260619 10:17:56.946719 10089 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781864224577058-825-0/minicluster-data/ts-0/wal/instance:
uuid: "2d880769cbd8414491ca90bdd6dbcc05"
format_stamp: "Formatted at 2026-06-19 10:17:56 on dist-test-slave-6pt2"
server_key: "892a6d607d7b1a0b9892fc8f93c41155"
server_key_iv: "1f4048c29724653fd7a0d04d4cc5b441"
server_key_version: "encryptionkey@0"
I20260619 10:17:56.948010 10089 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 10:17:56.948747 10103 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 10:17:56.948915 10089 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 10:17:56.949002 10089 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781864224577058-825-0/minicluster-data/ts-0/data,/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781864224577058-825-0/minicluster-data/ts-0/wal
uuid: "2d880769cbd8414491ca90bdd6dbcc05"
format_stamp: "Formatted at 2026-06-19 10:17:56 on dist-test-slave-6pt2"
server_key: "892a6d607d7b1a0b9892fc8f93c41155"
server_key_iv: "1f4048c29724653fd7a0d04d4cc5b441"
server_key_version: "encryptionkey@0"
I20260619 10:17:56.949064 10089 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781864224577058-825-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781864224577058-825-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781864224577058-825-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 10:17:56.965130 10089 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 10:17:56.965373 10089 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 10:17:56.965471 10089 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 10:17:56.965634 10089 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 10:17:56.965921 10089 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 10:17:56.965950 10089 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 10:17:56.965971 10089 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 10:17:56.965983 10089 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 10:17:56.971562 10089 rpc_server.cc:307] RPC server started. Bound to: 127.0.206.65:46779
I20260619 10:17:56.971638 10216 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.206.65:46779 every 8 connection(s)
I20260619 10:17:56.971958 10089 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781864224577058-825-0/minicluster-data/ts-0/data/info.pb
I20260619 10:17:56.973455 825 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTU9OP6/build/release/bin/kudu as pid 10089
I20260619 10:17:56.973552 825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781864224577058-825-0/minicluster-data/ts-0/wal/instance
I20260619 10:17:56.973765 825 external_mini_cluster.cc:1428] Setting key a300474a57513021b2b8d6a5b9ee3b7f
I20260619 10:17:56.974730 825 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTU9OP6/build/release/bin/kudu
/tmp/dist-test-taskTU9OP6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781864224577058-825-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781864224577058-825-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781864224577058-825-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781864224577058-825-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.206.66:0
--local_ip_for_outbound_sockets=127.0.206.66
--webserver_interface=127.0.206.66
--webserver_port=0
--tserver_master_addrs=127.0.206.126:44555
--builtin_ntp_servers=127.0.206.84:46029
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260619 10:17:56.977872 10069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:56.972662 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.65:41885 (local address 127.0.206.126:44555)
0619 10:17:56.972717 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0619 10:17:56.972719 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 10:17:56.972935 (+ 216us) server_negotiation.cc:407] Connection header received
0619 10:17:56.973534 (+ 599us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:17:56.973535 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:17:56.973549 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:17:56.973634 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:17:56.974507 (+ 873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:56.975163 (+ 656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:56.976491 (+ 1328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:56.976700 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:56.976782 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:56.977601 (+ 819us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 10:17:56.977602 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 10:17:56.977603 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 10:17:56.977608 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0619 10:17:56.977622 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 10:17:56.977626 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 10:17:56.977685 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0619 10:17:56.977776 (+ 91us) server_negotiation.cc:299] Negotiation successful
0619 10:17:56.977802 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260619 10:17:56.978072 10219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:56.972621 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:44555 (local address 127.0.206.65:41885)
0619 10:17:56.972802 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0619 10:17:56.972840 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0619 10:17:56.973417 (+ 577us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 10:17:56.973727 (+ 310us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 10:17:56.973732 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 10:17:56.973742 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0619 10:17:56.974417 (+ 675us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:56.974424 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:56.975251 (+ 827us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:56.975252 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:56.976391 (+ 1139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:56.976393 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:56.976792 (+ 399us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:56.976793 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:56.976922 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:56.977465 (+ 543us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 10:17:56.977465 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0619 10:17:56.977491 (+ 26us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 10:17:56.977512 (+ 21us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 10:17:56.977693 (+ 181us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 10:17:56.977697 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 10:17:56.977700 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 10:17:56.977778 (+ 78us) client_negotiation.cc:240] Negotiation successful
0619 10:17:56.977867 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":92,"threads_started":1}
I20260619 10:17:56.978492 10217 heartbeater.cc:347] Connected to a master server at 127.0.206.126:44555
I20260619 10:17:56.978572 10217 heartbeater.cc:464] Registering TS with master...
I20260619 10:17:56.978788 10217 heartbeater.cc:511] Master 127.0.206.126:44555 requested a full tablet report, sending...
I20260619 10:17:56.979312 10029 ts_manager.cc:194] Registered new tserver with Master: 2d880769cbd8414491ca90bdd6dbcc05 (127.0.206.65:46779)
I20260619 10:17:56.980294 10029 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.206.65:41885
W20260619 10:17:57.054958 10220 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 10:17:57.055141 10220 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 10:17:57.055167 10220 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 10:17:57.056233 10220 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 10:17:57.056264 10220 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 10:17:57.056288 10220 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.206.66
I20260619 10:17:57.057495 10220 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.206.84:46029
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781864224577058-825-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781864224577058-825-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.206.66:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781864224577058-825-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.206.66
--webserver_port=0
--tserver_master_addrs=127.0.206.126:44555
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10220
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.206.66
--log_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781864224577058-825-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 09:32:44 UTC on ccycloud.quasar-gbtldp.root.comops.site
build id 4455440
I20260619 10:17:57.057662 10220 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 10:17:57.057878 10220 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 10:17:57.062314 10225 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 10:17:57.062314 10228 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 10:17:57.062353 10226 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 10:17:57.062516 10220 server_base.cc:1034] running on GCE node
I20260619 10:17:57.062762 10220 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 10:17:57.062945 10220 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 10:17:57.064074 10220 hybrid_clock.cc:648] HybridClock initialized: now 1781864277064066 us; error 24 us; skew 500 ppm
I20260619 10:17:57.065223 10220 webserver.cc:460] Webserver started at http://127.0.206.66:44389/ using document root <none> and password file <none>
I20260619 10:17:57.065402 10220 fs_manager.cc:359] Metadata directory not provided
I20260619 10:17:57.065440 10220 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 10:17:57.065544 10220 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 10:17:57.066532 10220 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781864224577058-825-0/minicluster-data/ts-1/data/instance:
uuid: "6e1c2987499245908db3030d6770d6d2"
format_stamp: "Formatted at 2026-06-19 10:17:57 on dist-test-slave-6pt2"
server_key: "20a4cb1938d659519c9595c36decf428"
server_key_iv: "4d708918d140af1a266f90818761fe5d"
server_key_version: "encryptionkey@0"
I20260619 10:17:57.067008 10220 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781864224577058-825-0/minicluster-data/ts-1/wal/instance:
uuid: "6e1c2987499245908db3030d6770d6d2"
format_stamp: "Formatted at 2026-06-19 10:17:57 on dist-test-slave-6pt2"
server_key: "20a4cb1938d659519c9595c36decf428"
server_key_iv: "4d708918d140af1a266f90818761fe5d"
server_key_version: "encryptionkey@0"
I20260619 10:17:57.068360 10220 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.001s
I20260619 10:17:57.068976 10234 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 10:17:57.069164 10220 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 10:17:57.069257 10220 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781864224577058-825-0/minicluster-data/ts-1/data,/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781864224577058-825-0/minicluster-data/ts-1/wal
uuid: "6e1c2987499245908db3030d6770d6d2"
format_stamp: "Formatted at 2026-06-19 10:17:57 on dist-test-slave-6pt2"
server_key: "20a4cb1938d659519c9595c36decf428"
server_key_iv: "4d708918d140af1a266f90818761fe5d"
server_key_version: "encryptionkey@0"
I20260619 10:17:57.069308 10220 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781864224577058-825-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781864224577058-825-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781864224577058-825-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 10:17:57.078994 10220 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 10:17:57.079201 10220 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 10:17:57.079291 10220 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 10:17:57.079437 10220 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 10:17:57.079671 10220 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 10:17:57.079700 10220 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 10:17:57.079718 10220 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 10:17:57.079731 10220 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 10:17:57.084977 10220 rpc_server.cc:307] RPC server started. Bound to: 127.0.206.66:42335
I20260619 10:17:57.085047 10347 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.206.66:42335 every 8 connection(s)
I20260619 10:17:57.085356 10220 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781864224577058-825-0/minicluster-data/ts-1/data/info.pb
I20260619 10:17:57.089355 825 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTU9OP6/build/release/bin/kudu as pid 10220
I20260619 10:17:57.089447 825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781864224577058-825-0/minicluster-data/ts-1/wal/instance
I20260619 10:17:57.089612 825 external_mini_cluster.cc:1428] Setting key 0a8ee13312fc737bb6bfbfe947c6de02
I20260619 10:17:57.090132 10069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:57.086047 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.66:44301 (local address 127.0.206.126:44555)
0619 10:17:57.086124 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0619 10:17:57.086127 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 10:17:57.086320 (+ 193us) server_negotiation.cc:407] Connection header received
0619 10:17:57.086809 (+ 489us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:17:57.086810 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:17:57.086820 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:17:57.086870 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:17:57.087448 (+ 578us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:57.087923 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:57.088840 (+ 917us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:57.089015 (+ 175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:57.089063 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:57.089890 (+ 827us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 10:17:57.089891 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 10:17:57.089891 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 10:17:57.089896 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0619 10:17:57.089911 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 10:17:57.089916 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 10:17:57.089971 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0619 10:17:57.090057 (+ 86us) server_negotiation.cc:299] Negotiation successful
0619 10:17:57.090079 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260619 10:17:57.090404 10350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:57.086009 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:44555 (local address 127.0.206.66:44301)
0619 10:17:57.086211 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0619 10:17:57.086241 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0619 10:17:57.086727 (+ 486us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 10:17:57.086918 (+ 191us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 10:17:57.086921 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 10:17:57.086929 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0619 10:17:57.087384 (+ 455us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:57.087388 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:57.087996 (+ 608us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:57.087996 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:57.088748 (+ 752us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:57.088749 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:57.089080 (+ 331us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:57.089080 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:57.089202 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:57.089720 (+ 518us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 10:17:57.089721 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0619 10:17:57.089763 (+ 42us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 10:17:57.089792 (+ 29us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 10:17:57.089986 (+ 194us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 10:17:57.089991 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 10:17:57.089994 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 10:17:57.090059 (+ 65us) client_negotiation.cc:240] Negotiation successful
0619 10:17:57.090150 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":101,"threads_started":1}
I20260619 10:17:57.090699 825 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTU9OP6/build/release/bin/kudu
/tmp/dist-test-taskTU9OP6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781864224577058-825-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781864224577058-825-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781864224577058-825-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781864224577058-825-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.206.67:0
--local_ip_for_outbound_sockets=127.0.206.67
--webserver_interface=127.0.206.67
--webserver_port=0
--tserver_master_addrs=127.0.206.126:44555
--builtin_ntp_servers=127.0.206.84:46029
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260619 10:17:57.090967 10348 heartbeater.cc:347] Connected to a master server at 127.0.206.126:44555
I20260619 10:17:57.091054 10348 heartbeater.cc:464] Registering TS with master...
I20260619 10:17:57.091229 10348 heartbeater.cc:511] Master 127.0.206.126:44555 requested a full tablet report, sending...
I20260619 10:17:57.091586 10029 ts_manager.cc:194] Registered new tserver with Master: 6e1c2987499245908db3030d6770d6d2 (127.0.206.66:42335)
I20260619 10:17:57.092006 10029 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.206.66:44301
W20260619 10:17:57.168584 10351 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 10:17:57.168764 10351 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 10:17:57.168784 10351 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 10:17:57.169796 10351 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 10:17:57.169827 10351 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 10:17:57.169850 10351 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.206.67
I20260619 10:17:57.171186 10351 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.206.84:46029
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781864224577058-825-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781864224577058-825-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.206.67:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781864224577058-825-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.206.67
--webserver_port=0
--tserver_master_addrs=127.0.206.126:44555
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10351
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.206.67
--log_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781864224577058-825-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 09:32:44 UTC on ccycloud.quasar-gbtldp.root.comops.site
build id 4455440
I20260619 10:17:57.171423 10351 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 10:17:57.171697 10351 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 10:17:57.176324 10357 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 10:17:57.176330 10359 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 10:17:57.176381 10356 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 10:17:57.176653 10351 server_base.cc:1034] running on GCE node
I20260619 10:17:57.176872 10351 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 10:17:57.177125 10351 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 10:17:57.178282 10351 hybrid_clock.cc:648] HybridClock initialized: now 1781864277178268 us; error 27 us; skew 500 ppm
I20260619 10:17:57.179567 10351 webserver.cc:460] Webserver started at http://127.0.206.67:42181/ using document root <none> and password file <none>
I20260619 10:17:57.179816 10351 fs_manager.cc:359] Metadata directory not provided
I20260619 10:17:57.179888 10351 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 10:17:57.180039 10351 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 10:17:57.181087 10351 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781864224577058-825-0/minicluster-data/ts-2/data/instance:
uuid: "9777b59591064a49af1291d26665655d"
format_stamp: "Formatted at 2026-06-19 10:17:57 on dist-test-slave-6pt2"
server_key: "cf1145ca93554071784fbf954957466c"
server_key_iv: "a628b4a60cc1e5ec034bec9a51d07243"
server_key_version: "encryptionkey@0"
I20260619 10:17:57.181456 10351 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781864224577058-825-0/minicluster-data/ts-2/wal/instance:
uuid: "9777b59591064a49af1291d26665655d"
format_stamp: "Formatted at 2026-06-19 10:17:57 on dist-test-slave-6pt2"
server_key: "cf1145ca93554071784fbf954957466c"
server_key_iv: "a628b4a60cc1e5ec034bec9a51d07243"
server_key_version: "encryptionkey@0"
I20260619 10:17:57.182729 10351 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260619 10:17:57.183451 10365 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 10:17:57.183614 10351 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 10:17:57.183708 10351 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781864224577058-825-0/minicluster-data/ts-2/data,/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781864224577058-825-0/minicluster-data/ts-2/wal
uuid: "9777b59591064a49af1291d26665655d"
format_stamp: "Formatted at 2026-06-19 10:17:57 on dist-test-slave-6pt2"
server_key: "cf1145ca93554071784fbf954957466c"
server_key_iv: "a628b4a60cc1e5ec034bec9a51d07243"
server_key_version: "encryptionkey@0"
I20260619 10:17:57.183794 10351 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781864224577058-825-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781864224577058-825-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781864224577058-825-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 10:17:57.204247 10351 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 10:17:57.204562 10351 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 10:17:57.204711 10351 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 10:17:57.204921 10351 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 10:17:57.205233 10351 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 10:17:57.205288 10351 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 10:17:57.205328 10351 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 10:17:57.205356 10351 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 10:17:57.210896 10351 rpc_server.cc:307] RPC server started. Bound to: 127.0.206.67:44437
I20260619 10:17:57.210947 10478 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.206.67:44437 every 8 connection(s)
I20260619 10:17:57.211355 10351 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781864224577058-825-0/minicluster-data/ts-2/data/info.pb
I20260619 10:17:57.215713 825 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTU9OP6/build/release/bin/kudu as pid 10351
I20260619 10:17:57.215807 825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781864224577058-825-0/minicluster-data/ts-2/wal/instance
I20260619 10:17:57.215993 825 external_mini_cluster.cc:1428] Setting key e53b6fe0b97f6a5b526595bf637d6c46
I20260619 10:17:57.216444 10069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:57.212040 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.67:56809 (local address 127.0.206.126:44555)
0619 10:17:57.212105 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0619 10:17:57.212107 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 10:17:57.212385 (+ 278us) server_negotiation.cc:407] Connection header received
0619 10:17:57.212836 (+ 451us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:17:57.212837 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:17:57.212847 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:17:57.212893 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:17:57.213518 (+ 625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:57.213992 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:57.215000 (+ 1008us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:57.215185 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:57.215238 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:57.216109 (+ 871us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 10:17:57.216110 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 10:17:57.216111 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 10:17:57.216115 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0619 10:17:57.216129 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 10:17:57.216135 (+ 6us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 10:17:57.216192 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0619 10:17:57.216309 (+ 117us) server_negotiation.cc:299] Negotiation successful
0619 10:17:57.216338 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260619 10:17:57.216646 10481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:57.212061 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:44555 (local address 127.0.206.67:56809)
0619 10:17:57.212255 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0619 10:17:57.212287 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0619 10:17:57.212758 (+ 471us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 10:17:57.212968 (+ 210us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 10:17:57.212971 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 10:17:57.212979 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0619 10:17:57.213452 (+ 473us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:57.213456 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:57.214067 (+ 611us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:57.214068 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:57.214910 (+ 842us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:57.214912 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:57.215255 (+ 343us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:57.215255 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:57.215356 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:57.215937 (+ 581us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 10:17:57.215937 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0619 10:17:57.215973 (+ 36us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 10:17:57.215994 (+ 21us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 10:17:57.216220 (+ 226us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 10:17:57.216225 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 10:17:57.216228 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 10:17:57.216295 (+ 67us) client_negotiation.cc:240] Negotiation successful
0619 10:17:57.216416 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":98,"threads_started":1}
I20260619 10:17:57.217022 10479 heartbeater.cc:347] Connected to a master server at 127.0.206.126:44555
I20260619 10:17:57.217128 10479 heartbeater.cc:464] Registering TS with master...
I20260619 10:17:57.217306 10479 heartbeater.cc:511] Master 127.0.206.126:44555 requested a full tablet report, sending...
I20260619 10:17:57.217646 10029 ts_manager.cc:194] Registered new tserver with Master: 9777b59591064a49af1291d26665655d (127.0.206.67:44437)
I20260619 10:17:57.217947 10029 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.206.67:56809
I20260619 10:17:57.227841 825 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260619 10:17:57.229506 10069 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 10:17:57.229052 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49836 (local address 127.0.206.126:44555)
0619 10:17:57.229121 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0619 10:17:57.229123 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 10:17:57.229201 (+ 78us) server_negotiation.cc:407] Connection header received
0619 10:17:57.229234 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:17:57.229235 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:17:57.229245 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:17:57.229296 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:17:57.229441 (+ 145us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49836: BlockingRecv error: recv got EOF from 127.0.0.1:49836 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
I20260619 10:17:57.229558 825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTU9OP6/build/release/bin/kudu with pid 10089
I20260619 10:17:57.234107 825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTU9OP6/build/release/bin/kudu with pid 10220
I20260619 10:17:57.238096 825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTU9OP6/build/release/bin/kudu with pid 10351
I20260619 10:17:57.241865 825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTU9OP6/build/release/bin/kudu with pid 9998
2026-06-19T10:17:57Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (572 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-06-19T10:17:57Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T10:17:57Z Disabled control of system clock
I20260619 10:17:57.267391 825 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTU9OP6/build/release/bin/kudu
/tmp/dist-test-taskTU9OP6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781864224577058-825-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781864224577058-825-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781864224577058-825-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781864224577058-825-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.206.126:41343
--webserver_interface=127.0.206.126
--webserver_port=0
--builtin_ntp_servers=127.0.206.84:37711
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.206.126:41343
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260619 10:17:57.346004 10490 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 10:17:57.346210 10490 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 10:17:57.346230 10490 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 10:17:57.347314 10490 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 10:17:57.347347 10490 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 10:17:57.347359 10490 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 10:17:57.347371 10490 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 10:17:57.347383 10490 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 10:17:57.348614 10490 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.206.84:37711
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781864224577058-825-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781864224577058-825-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.206.126:41343
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.206.126:41343
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781864224577058-825-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.206.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10490
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781864224577058-825-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 09:32:44 UTC on ccycloud.quasar-gbtldp.root.comops.site
build id 4455440
I20260619 10:17:57.348798 10490 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 10:17:57.349009 10490 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 10:17:57.353781 10498 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 10:17:57.353778 10496 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 10:17:57.353826 10495 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 10:17:57.353843 10490 server_base.cc:1034] running on GCE node
I20260619 10:17:57.354189 10490 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 10:17:57.354455 10490 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 10:17:57.355593 10490 hybrid_clock.cc:648] HybridClock initialized: now 1781864277355570 us; error 37 us; skew 500 ppm
I20260619 10:17:57.356760 10490 webserver.cc:460] Webserver started at http://127.0.206.126:42155/ using document root <none> and password file <none>
I20260619 10:17:57.357000 10490 fs_manager.cc:359] Metadata directory not provided
I20260619 10:17:57.357045 10490 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 10:17:57.357205 10490 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 10:17:57.358177 10490 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781864224577058-825-0/minicluster-data/master-0/data/instance:
uuid: "1cf56242a4764817ad00ab49d2aec607"
format_stamp: "Formatted at 2026-06-19 10:17:57 on dist-test-slave-6pt2"
server_key: "afc393860012fc705b246d74707b8ac3"
server_key_iv: "ed3d7a8e75ac78df4d5a43a3c074f5f9"
server_key_version: "encryptionkey@0"
I20260619 10:17:57.358515 10490 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781864224577058-825-0/minicluster-data/master-0/wal/instance:
uuid: "1cf56242a4764817ad00ab49d2aec607"
format_stamp: "Formatted at 2026-06-19 10:17:57 on dist-test-slave-6pt2"
server_key: "afc393860012fc705b246d74707b8ac3"
server_key_iv: "ed3d7a8e75ac78df4d5a43a3c074f5f9"
server_key_version: "encryptionkey@0"
I20260619 10:17:57.359701 10490 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260619 10:17:57.360328 10504 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 10:17:57.360497 10490 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260619 10:17:57.360582 10490 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781864224577058-825-0/minicluster-data/master-0/data,/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781864224577058-825-0/minicluster-data/master-0/wal
uuid: "1cf56242a4764817ad00ab49d2aec607"
format_stamp: "Formatted at 2026-06-19 10:17:57 on dist-test-slave-6pt2"
server_key: "afc393860012fc705b246d74707b8ac3"
server_key_iv: "ed3d7a8e75ac78df4d5a43a3c074f5f9"
server_key_version: "encryptionkey@0"
I20260619 10:17:57.360661 10490 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781864224577058-825-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781864224577058-825-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781864224577058-825-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 10:17:57.363826 10490 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 10:17:57.364060 10490 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 10:17:57.364197 10490 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 10:17:57.367789 10490 rpc_server.cc:307] RPC server started. Bound to: 127.0.206.126:41343
I20260619 10:17:57.367815 10556 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.206.126:41343 every 8 connection(s)
I20260619 10:17:57.368106 10490 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781864224577058-825-0/minicluster-data/master-0/data/info.pb
I20260619 10:17:57.368674 10557 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 10:17:57.371145 825 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTU9OP6/build/release/bin/kudu as pid 10490
I20260619 10:17:57.371162 10557 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1cf56242a4764817ad00ab49d2aec607: Bootstrap starting.
I20260619 10:17:57.371245 825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781864224577058-825-0/minicluster-data/master-0/wal/instance
I20260619 10:17:57.371403 825 external_mini_cluster.cc:1428] Setting key 85e9b9ac2a38d65a710e475e5a51a0e9
I20260619 10:17:57.371810 10557 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1cf56242a4764817ad00ab49d2aec607: Neither blocks nor log segments found. Creating new log.
I20260619 10:17:57.372233 10557 log.cc:824] T 00000000000000000000000000000000 P 1cf56242a4764817ad00ab49d2aec607: Log is configured to *not* fsync() on all Append() calls
I20260619 10:17:57.372957 10557 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1cf56242a4764817ad00ab49d2aec607: No bootstrap required, opened a new log
I20260619 10:17:57.373888 10560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:57.371890 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58690 (local address 127.0.206.126:41343)
0619 10:17:57.372105 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0619 10:17:57.372113 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 10:17:57.372149 (+ 36us) server_negotiation.cc:407] Connection header received
0619 10:17:57.372681 (+ 532us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:17:57.372690 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:17:57.372741 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:17:57.372888 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:17:57.373483 (+ 595us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 10:17:57.373486 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 10:17:57.373487 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 10:17:57.373491 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0619 10:17:57.373533 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 10:17:57.373546 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 10:17:57.373622 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0619 10:17:57.373656 (+ 34us) server_negotiation.cc:299] Negotiation successful
0619 10:17:57.373709 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":129,"thread_start_us":77,"threads_started":1}
I20260619 10:17:57.375022 10557 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1cf56242a4764817ad00ab49d2aec607 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1cf56242a4764817ad00ab49d2aec607" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 41343 } }
I20260619 10:17:57.375145 10557 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1cf56242a4764817ad00ab49d2aec607 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 10:17:57.375167 10557 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1cf56242a4764817ad00ab49d2aec607 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1cf56242a4764817ad00ab49d2aec607, State: Initialized, Role: FOLLOWER
I20260619 10:17:57.375276 10557 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1cf56242a4764817ad00ab49d2aec607 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1cf56242a4764817ad00ab49d2aec607" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 41343 } }
I20260619 10:17:57.375355 10557 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1cf56242a4764817ad00ab49d2aec607 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 10:17:57.375378 10557 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1cf56242a4764817ad00ab49d2aec607 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 10:17:57.375406 10557 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1cf56242a4764817ad00ab49d2aec607 [term 0 FOLLOWER]: Advancing to term 1
I20260619 10:17:57.376142 10557 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1cf56242a4764817ad00ab49d2aec607 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1cf56242a4764817ad00ab49d2aec607" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 41343 } }
I20260619 10:17:57.376222 10557 leader_election.cc:302] T 00000000000000000000000000000000 P 1cf56242a4764817ad00ab49d2aec607 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1cf56242a4764817ad00ab49d2aec607; no voters:
I20260619 10:17:57.376331 10557 leader_election.cc:288] T 00000000000000000000000000000000 P 1cf56242a4764817ad00ab49d2aec607 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 10:17:57.376411 10562 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1cf56242a4764817ad00ab49d2aec607 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 10:17:57.376582 10562 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1cf56242a4764817ad00ab49d2aec607 [term 1 LEADER]: Becoming Leader. State: Replica: 1cf56242a4764817ad00ab49d2aec607, State: Running, Role: LEADER
I20260619 10:17:57.376607 10557 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1cf56242a4764817ad00ab49d2aec607 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 10:17:57.376677 10562 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1cf56242a4764817ad00ab49d2aec607 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1cf56242a4764817ad00ab49d2aec607" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 41343 } }
I20260619 10:17:57.377038 10564 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1cf56242a4764817ad00ab49d2aec607 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1cf56242a4764817ad00ab49d2aec607. Latest consensus state: current_term: 1 leader_uuid: "1cf56242a4764817ad00ab49d2aec607" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1cf56242a4764817ad00ab49d2aec607" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 41343 } } }
I20260619 10:17:57.377128 10564 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1cf56242a4764817ad00ab49d2aec607 [sys.catalog]: This master's current role is: LEADER
I20260619 10:17:57.377329 10571 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 10:17:57.377339 10563 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1cf56242a4764817ad00ab49d2aec607 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1cf56242a4764817ad00ab49d2aec607" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1cf56242a4764817ad00ab49d2aec607" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 41343 } } }
I20260619 10:17:57.377592 10563 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1cf56242a4764817ad00ab49d2aec607 [sys.catalog]: This master's current role is: LEADER
I20260619 10:17:57.377939 10571 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 10:17:57.379390 10571 catalog_manager.cc:1347] Generated new cluster ID: 15dd44ff9a874622ac5ae67854d00b2e
I20260619 10:17:57.379441 10571 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 10:17:57.393937 10571 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 10:17:57.394109 10571 catalog_manager.cc:1504] Loading token signing keys...
I20260619 10:17:57.402712 10571 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1cf56242a4764817ad00ab49d2aec607: Generated new TSK 0
I20260619 10:17:57.402900 10571 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 10:17:57.410799 825 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTU9OP6/build/release/bin/kudu
/tmp/dist-test-taskTU9OP6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781864224577058-825-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781864224577058-825-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781864224577058-825-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781864224577058-825-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.206.65:0
--local_ip_for_outbound_sockets=127.0.206.65
--webserver_interface=127.0.206.65
--webserver_port=0
--tserver_master_addrs=127.0.206.126:41343
--builtin_ntp_servers=127.0.206.84:37711
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260619 10:17:57.490752 10581 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 10:17:57.490947 10581 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 10:17:57.490968 10581 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 10:17:57.492074 10581 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 10:17:57.492110 10581 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 10:17:57.492133 10581 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.206.65
I20260619 10:17:57.493386 10581 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.206.84:37711
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781864224577058-825-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781864224577058-825-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.206.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781864224577058-825-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.206.65
--webserver_port=0
--tserver_master_addrs=127.0.206.126:41343
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10581
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.206.65
--log_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781864224577058-825-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 09:32:44 UTC on ccycloud.quasar-gbtldp.root.comops.site
build id 4455440
I20260619 10:17:57.493602 10581 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 10:17:57.493875 10581 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 10:17:57.498827 10587 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 10:17:57.498884 10586 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 10:17:57.498826 10589 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 10:17:57.499014 10581 server_base.cc:1034] running on GCE node
I20260619 10:17:57.499254 10581 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 10:17:57.499498 10581 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 10:17:57.500666 10581 hybrid_clock.cc:648] HybridClock initialized: now 1781864277500648 us; error 30 us; skew 500 ppm
I20260619 10:17:57.501822 10581 webserver.cc:460] Webserver started at http://127.0.206.65:41939/ using document root <none> and password file <none>
I20260619 10:17:57.502048 10581 fs_manager.cc:359] Metadata directory not provided
I20260619 10:17:57.502111 10581 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 10:17:57.502249 10581 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 10:17:57.503293 10581 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781864224577058-825-0/minicluster-data/ts-0/data/instance:
uuid: "2626673c9ec248718278f84ac366ba9f"
format_stamp: "Formatted at 2026-06-19 10:17:57 on dist-test-slave-6pt2"
server_key: "178ee9ea239e3d96249be43dbe1f0562"
server_key_iv: "7895b779afc49021f71b4a520ec6dfe9"
server_key_version: "encryptionkey@0"
I20260619 10:17:57.503636 10581 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781864224577058-825-0/minicluster-data/ts-0/wal/instance:
uuid: "2626673c9ec248718278f84ac366ba9f"
format_stamp: "Formatted at 2026-06-19 10:17:57 on dist-test-slave-6pt2"
server_key: "178ee9ea239e3d96249be43dbe1f0562"
server_key_iv: "7895b779afc49021f71b4a520ec6dfe9"
server_key_version: "encryptionkey@0"
I20260619 10:17:57.504865 10581 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 10:17:57.505554 10595 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 10:17:57.505722 10581 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 10:17:57.505815 10581 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781864224577058-825-0/minicluster-data/ts-0/data,/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781864224577058-825-0/minicluster-data/ts-0/wal
uuid: "2626673c9ec248718278f84ac366ba9f"
format_stamp: "Formatted at 2026-06-19 10:17:57 on dist-test-slave-6pt2"
server_key: "178ee9ea239e3d96249be43dbe1f0562"
server_key_iv: "7895b779afc49021f71b4a520ec6dfe9"
server_key_version: "encryptionkey@0"
I20260619 10:17:57.505897 10581 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781864224577058-825-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781864224577058-825-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781864224577058-825-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 10:17:57.509097 10581 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 10:17:57.509336 10581 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 10:17:57.509462 10581 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 10:17:57.509660 10581 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 10:17:57.509941 10581 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 10:17:57.509992 10581 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 10:17:57.510028 10581 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 10:17:57.510056 10581 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 10:17:57.515779 10581 rpc_server.cc:307] RPC server started. Bound to: 127.0.206.65:38075
I20260619 10:17:57.515810 10708 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.206.65:38075 every 8 connection(s)
I20260619 10:17:57.516094 10581 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781864224577058-825-0/minicluster-data/ts-0/data/info.pb
I20260619 10:17:57.518554 10560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:57.516716 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.65:56453 (local address 127.0.206.126:41343)
0619 10:17:57.516787 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0619 10:17:57.516790 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 10:17:57.516955 (+ 165us) server_negotiation.cc:407] Connection header received
0619 10:17:57.517347 (+ 392us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:17:57.517348 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:17:57.517352 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:17:57.517402 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:17:57.518380 (+ 978us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 10:17:57.518381 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 10:17:57.518381 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 10:17:57.518385 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0619 10:17:57.518397 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 10:17:57.518401 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 10:17:57.518444 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0619 10:17:57.518495 (+ 51us) server_negotiation.cc:299] Negotiation successful
0619 10:17:57.518506 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260619 10:17:57.518782 10711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:57.516674 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:41343 (local address 127.0.206.65:56453)
0619 10:17:57.516849 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0619 10:17:57.516879 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0619 10:17:57.517273 (+ 394us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 10:17:57.517445 (+ 172us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 10:17:57.517447 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 10:17:57.517455 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0619 10:17:57.518255 (+ 800us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 10:17:57.518256 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0619 10:17:57.518279 (+ 23us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 10:17:57.518304 (+ 25us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 10:17:57.518442 (+ 138us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 10:17:57.518444 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 10:17:57.518446 (+ 2us) client_negotiation.cc:769] Sending connection context
0619 10:17:57.518494 (+ 48us) client_negotiation.cc:240] Negotiation successful
0619 10:17:57.518535 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":93,"threads_started":1}
I20260619 10:17:57.519336 10709 heartbeater.cc:347] Connected to a master server at 127.0.206.126:41343
I20260619 10:17:57.519433 10709 heartbeater.cc:464] Registering TS with master...
I20260619 10:17:57.519563 10709 heartbeater.cc:511] Master 127.0.206.126:41343 requested a full tablet report, sending...
I20260619 10:17:57.520018 10521 ts_manager.cc:194] Registered new tserver with Master: 2626673c9ec248718278f84ac366ba9f (127.0.206.65:38075)
I20260619 10:17:57.525450 825 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTU9OP6/build/release/bin/kudu as pid 10581
I20260619 10:17:57.525522 825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781864224577058-825-0/minicluster-data/ts-0/wal/instance
I20260619 10:17:57.525702 825 external_mini_cluster.cc:1428] Setting key 3da4c3c009b417bc0eb1ce1794352f48
I20260619 10:17:57.526551 825 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTU9OP6/build/release/bin/kudu
/tmp/dist-test-taskTU9OP6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781864224577058-825-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781864224577058-825-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781864224577058-825-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781864224577058-825-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.206.66:0
--local_ip_for_outbound_sockets=127.0.206.66
--webserver_interface=127.0.206.66
--webserver_port=0
--tserver_master_addrs=127.0.206.126:41343
--builtin_ntp_servers=127.0.206.84:37711
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260619 10:17:57.603708 10712 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 10:17:57.603917 10712 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 10:17:57.603950 10712 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 10:17:57.605173 10712 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 10:17:57.605228 10712 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 10:17:57.605268 10712 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.206.66
I20260619 10:17:57.606606 10712 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.206.84:37711
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781864224577058-825-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781864224577058-825-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.206.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781864224577058-825-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.206.66
--webserver_port=0
--tserver_master_addrs=127.0.206.126:41343
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10712
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.206.66
--log_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781864224577058-825-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 09:32:44 UTC on ccycloud.quasar-gbtldp.root.comops.site
build id 4455440
I20260619 10:17:57.606909 10712 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 10:17:57.607183 10712 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 10:17:57.611963 10717 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 10:17:57.611977 10718 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 10:17:57.611977 10720 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 10:17:57.612134 10712 server_base.cc:1034] running on GCE node
I20260619 10:17:57.612376 10712 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 10:17:57.612636 10712 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 10:17:57.613797 10712 hybrid_clock.cc:648] HybridClock initialized: now 1781864277613778 us; error 33 us; skew 500 ppm
I20260619 10:17:57.617929 10712 webserver.cc:460] Webserver started at http://127.0.206.66:38411/ using document root <none> and password file <none>
I20260619 10:17:57.618180 10712 fs_manager.cc:359] Metadata directory not provided
I20260619 10:17:57.618260 10712 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 10:17:57.618407 10712 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 10:17:57.619531 10712 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781864224577058-825-0/minicluster-data/ts-1/data/instance:
uuid: "e41fd0a6a4ee4bcfa2177eac9985a1d0"
format_stamp: "Formatted at 2026-06-19 10:17:57 on dist-test-slave-6pt2"
server_key: "faab36a62f014cf648954e27d3a6c062"
server_key_iv: "a99f1709982c7e97f40e49a589e87115"
server_key_version: "encryptionkey@0"
I20260619 10:17:57.619906 10712 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781864224577058-825-0/minicluster-data/ts-1/wal/instance:
uuid: "e41fd0a6a4ee4bcfa2177eac9985a1d0"
format_stamp: "Formatted at 2026-06-19 10:17:57 on dist-test-slave-6pt2"
server_key: "faab36a62f014cf648954e27d3a6c062"
server_key_iv: "a99f1709982c7e97f40e49a589e87115"
server_key_version: "encryptionkey@0"
I20260619 10:17:57.621155 10712 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 10:17:57.621836 10726 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 10:17:57.622030 10712 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 10:17:57.622110 10712 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781864224577058-825-0/minicluster-data/ts-1/data,/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781864224577058-825-0/minicluster-data/ts-1/wal
uuid: "e41fd0a6a4ee4bcfa2177eac9985a1d0"
format_stamp: "Formatted at 2026-06-19 10:17:57 on dist-test-slave-6pt2"
server_key: "faab36a62f014cf648954e27d3a6c062"
server_key_iv: "a99f1709982c7e97f40e49a589e87115"
server_key_version: "encryptionkey@0"
I20260619 10:17:57.622192 10712 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781864224577058-825-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781864224577058-825-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781864224577058-825-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 10:17:57.625456 10712 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 10:17:57.625708 10712 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 10:17:57.625835 10712 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 10:17:57.626047 10712 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 10:17:57.626336 10712 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 10:17:57.626367 10712 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 10:17:57.626418 10712 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 10:17:57.626446 10712 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 10:17:57.632074 10712 rpc_server.cc:307] RPC server started. Bound to: 127.0.206.66:33977
I20260619 10:17:57.632114 10839 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.206.66:33977 every 8 connection(s)
I20260619 10:17:57.632445 10712 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781864224577058-825-0/minicluster-data/ts-1/data/info.pb
I20260619 10:17:57.635190 10560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:57.633151 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.66:51531 (local address 127.0.206.126:41343)
0619 10:17:57.633212 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0619 10:17:57.633214 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 10:17:57.633436 (+ 222us) server_negotiation.cc:407] Connection header received
0619 10:17:57.633850 (+ 414us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:17:57.633851 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:17:57.633857 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:17:57.633908 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:17:57.634994 (+ 1086us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 10:17:57.634996 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 10:17:57.634996 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 10:17:57.635000 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0619 10:17:57.635014 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 10:17:57.635017 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 10:17:57.635078 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0619 10:17:57.635127 (+ 49us) server_negotiation.cc:299] Negotiation successful
0619 10:17:57.635139 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260619 10:17:57.635365 10842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:57.633148 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:41343 (local address 127.0.206.66:51531)
0619 10:17:57.633330 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0619 10:17:57.633359 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0619 10:17:57.633776 (+ 417us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 10:17:57.633953 (+ 177us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 10:17:57.633956 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 10:17:57.633963 (+ 7us) client_negotiation.cc:189] Negotiated authn=SASL
0619 10:17:57.634857 (+ 894us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 10:17:57.634858 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0619 10:17:57.634883 (+ 25us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 10:17:57.634908 (+ 25us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 10:17:57.635079 (+ 171us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 10:17:57.635081 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 10:17:57.635083 (+ 2us) client_negotiation.cc:769] Sending connection context
0619 10:17:57.635134 (+ 51us) client_negotiation.cc:240] Negotiation successful
0619 10:17:57.635177 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":94,"threads_started":1}
I20260619 10:17:57.635825 10840 heartbeater.cc:347] Connected to a master server at 127.0.206.126:41343
I20260619 10:17:57.635895 10840 heartbeater.cc:464] Registering TS with master...
I20260619 10:17:57.635996 10840 heartbeater.cc:511] Master 127.0.206.126:41343 requested a full tablet report, sending...
I20260619 10:17:57.636286 10521 ts_manager.cc:194] Registered new tserver with Master: e41fd0a6a4ee4bcfa2177eac9985a1d0 (127.0.206.66:33977)
I20260619 10:17:57.640743 825 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTU9OP6/build/release/bin/kudu as pid 10712
I20260619 10:17:57.640843 825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781864224577058-825-0/minicluster-data/ts-1/wal/instance
I20260619 10:17:57.641042 825 external_mini_cluster.cc:1428] Setting key d0811c8c052b66dc62bf640df98cea48
I20260619 10:17:57.641805 825 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTU9OP6/build/release/bin/kudu
/tmp/dist-test-taskTU9OP6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781864224577058-825-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781864224577058-825-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781864224577058-825-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781864224577058-825-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.206.67:0
--local_ip_for_outbound_sockets=127.0.206.67
--webserver_interface=127.0.206.67
--webserver_port=0
--tserver_master_addrs=127.0.206.126:41343
--builtin_ntp_servers=127.0.206.84:37711
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260619 10:17:57.720878 10843 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 10:17:57.721109 10843 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 10:17:57.721130 10843 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 10:17:57.722306 10843 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 10:17:57.722347 10843 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 10:17:57.722400 10843 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.206.67
I20260619 10:17:57.723763 10843 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.206.84:37711
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781864224577058-825-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781864224577058-825-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.206.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781864224577058-825-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.206.67
--webserver_port=0
--tserver_master_addrs=127.0.206.126:41343
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10843
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.206.67
--log_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781864224577058-825-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 09:32:44 UTC on ccycloud.quasar-gbtldp.root.comops.site
build id 4455440
I20260619 10:17:57.724016 10843 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 10:17:57.724319 10843 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 10:17:57.729130 10849 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 10:17:57.729146 10848 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 10:17:57.729140 10851 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 10:17:57.729296 10843 server_base.cc:1034] running on GCE node
I20260619 10:17:57.729537 10843 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 10:17:57.729784 10843 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 10:17:57.730939 10843 hybrid_clock.cc:648] HybridClock initialized: now 1781864277730931 us; error 31 us; skew 500 ppm
I20260619 10:17:57.732117 10843 webserver.cc:460] Webserver started at http://127.0.206.67:33463/ using document root <none> and password file <none>
I20260619 10:17:57.732338 10843 fs_manager.cc:359] Metadata directory not provided
I20260619 10:17:57.732402 10843 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 10:17:57.732546 10843 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 10:17:57.733560 10843 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781864224577058-825-0/minicluster-data/ts-2/data/instance:
uuid: "5a7e4ff712234746b5bfbff66eec4786"
format_stamp: "Formatted at 2026-06-19 10:17:57 on dist-test-slave-6pt2"
server_key: "6e454686798acc117937ec65d95d3947"
server_key_iv: "cce3df91c35553851601eed6bbd9d0ed"
server_key_version: "encryptionkey@0"
I20260619 10:17:57.733917 10843 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781864224577058-825-0/minicluster-data/ts-2/wal/instance:
uuid: "5a7e4ff712234746b5bfbff66eec4786"
format_stamp: "Formatted at 2026-06-19 10:17:57 on dist-test-slave-6pt2"
server_key: "6e454686798acc117937ec65d95d3947"
server_key_iv: "cce3df91c35553851601eed6bbd9d0ed"
server_key_version: "encryptionkey@0"
I20260619 10:17:57.735251 10843 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 10:17:57.735965 10857 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 10:17:57.736163 10843 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 10:17:57.736267 10843 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781864224577058-825-0/minicluster-data/ts-2/data,/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781864224577058-825-0/minicluster-data/ts-2/wal
uuid: "5a7e4ff712234746b5bfbff66eec4786"
format_stamp: "Formatted at 2026-06-19 10:17:57 on dist-test-slave-6pt2"
server_key: "6e454686798acc117937ec65d95d3947"
server_key_iv: "cce3df91c35553851601eed6bbd9d0ed"
server_key_version: "encryptionkey@0"
I20260619 10:17:57.736349 10843 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781864224577058-825-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781864224577058-825-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781864224577058-825-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 10:17:57.739539 10843 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 10:17:57.739780 10843 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 10:17:57.739918 10843 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 10:17:57.740115 10843 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 10:17:57.740411 10843 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 10:17:57.740463 10843 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 10:17:57.740500 10843 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 10:17:57.740528 10843 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 10:17:57.745855 10843 rpc_server.cc:307] RPC server started. Bound to: 127.0.206.67:40501
I20260619 10:17:57.745924 10970 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.206.67:40501 every 8 connection(s)
I20260619 10:17:57.746234 10843 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781864224577058-825-0/minicluster-data/ts-2/data/info.pb
I20260619 10:17:57.748757 10560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:57.746927 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.67:40613 (local address 127.0.206.126:41343)
0619 10:17:57.746966 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0619 10:17:57.746968 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 10:17:57.747187 (+ 219us) server_negotiation.cc:407] Connection header received
0619 10:17:57.747567 (+ 380us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:17:57.747568 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:17:57.747572 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:17:57.747626 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:17:57.748586 (+ 960us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 10:17:57.748588 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 10:17:57.748588 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 10:17:57.748591 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0619 10:17:57.748603 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 10:17:57.748607 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 10:17:57.748650 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0619 10:17:57.748693 (+ 43us) server_negotiation.cc:299] Negotiation successful
0619 10:17:57.748710 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260619 10:17:57.748946 10973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:57.746883 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:41343 (local address 127.0.206.67:40613)
0619 10:17:57.747054 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0619 10:17:57.747091 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0619 10:17:57.747493 (+ 402us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 10:17:57.747672 (+ 179us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 10:17:57.747674 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 10:17:57.747682 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0619 10:17:57.748464 (+ 782us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 10:17:57.748464 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0619 10:17:57.748489 (+ 25us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 10:17:57.748512 (+ 23us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 10:17:57.748647 (+ 135us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 10:17:57.748649 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 10:17:57.748652 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 10:17:57.748700 (+ 48us) client_negotiation.cc:240] Negotiation successful
0619 10:17:57.748741 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":86,"threads_started":1}
I20260619 10:17:57.749408 10971 heartbeater.cc:347] Connected to a master server at 127.0.206.126:41343
I20260619 10:17:57.749486 10971 heartbeater.cc:464] Registering TS with master...
I20260619 10:17:57.749620 10971 heartbeater.cc:511] Master 127.0.206.126:41343 requested a full tablet report, sending...
I20260619 10:17:57.749945 10521 ts_manager.cc:194] Registered new tserver with Master: 5a7e4ff712234746b5bfbff66eec4786 (127.0.206.67:40501)
I20260619 10:17:57.756126 825 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTU9OP6/build/release/bin/kudu as pid 10843
I20260619 10:17:57.756199 825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781864224577058-825-0/minicluster-data/ts-2/wal/instance
I20260619 10:17:57.756335 825 external_mini_cluster.cc:1428] Setting key 446f6cac53a0e63b531dc64ff377136d
I20260619 10:17:57.757162 825 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260619 10:17:57.758888 10560 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 10:17:57.758392 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58706 (local address 127.0.206.126:41343)
0619 10:17:57.758447 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0619 10:17:57.758449 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 10:17:57.758565 (+ 116us) server_negotiation.cc:407] Connection header received
0619 10:17:57.758666 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:17:57.758667 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:17:57.758671 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:17:57.758730 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:17:57.758831 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58706: BlockingRecv error: recv got EOF from 127.0.0.1:58706 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
I20260619 10:17:57.759027 825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTU9OP6/build/release/bin/kudu with pid 10581
I20260619 10:17:57.763736 825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTU9OP6/build/release/bin/kudu with pid 10712
I20260619 10:17:57.768174 825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTU9OP6/build/release/bin/kudu with pid 10843
I20260619 10:17:57.772339 825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTU9OP6/build/release/bin/kudu with pid 10490
2026-06-19T10:17:57Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (530 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781864224577058-825-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 10:17:57 dist-test-slave-6pt2 krb5kdc[10981](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 10:17:57 dist-test-slave-6pt2 krb5kdc[10981](info): set up 2 sockets
Jun 19 10:17:57 dist-test-slave-6pt2 krb5kdc[10981](info): commencing operation
krb5kdc: starting...
W20260619 10:17:58.819509 825 mini_kdc.cc:121] Time spent starting KDC: real 1.029s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781864224577058-825-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781864224577058-825-0/krb5kdc/test-user.keytab.
Jun 19 10:17:58 dist-test-slave-6pt2 krb5kdc[10981](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864278, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T10:17:58Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T10:17:58Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.206.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.206.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.206.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781864224577058-825-0/krb5kdc/kudu_127.0.206.126.keytab.
Entry for principal kudu/127.0.206.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781864224577058-825-0/krb5kdc/kudu_127.0.206.126.keytab.
I20260619 10:17:58.928541 825 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTU9OP6/build/release/bin/kudu
/tmp/dist-test-taskTU9OP6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781864224577058-825-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781864224577058-825-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781864224577058-825-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781864224577058-825-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.206.126:32973
--webserver_interface=127.0.206.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781864224577058-825-0/krb5kdc/kudu_127.0.206.126.keytab
--principal=kudu/127.0.206.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.206.84:36379
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.206.126:32973
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781864224577058-825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781864224577058-825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781864224577058-825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 10:17:59.006632 10994 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 10:17:59.006819 10994 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 10:17:59.006839 10994 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 10:17:59.007988 10994 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 10:17:59.008029 10994 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 10:17:59.008042 10994 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 10:17:59.008054 10994 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 10:17:59.008111 10994 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 10:17:59.009476 10994 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.206.84:36379
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781864224577058-825-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781864224577058-825-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.206.126:32973
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781864224577058-825-0/krb5kdc/kudu_127.0.206.126.keytab
--principal=kudu/127.0.206.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.206.126:32973
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781864224577058-825-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.206.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10994
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781864224577058-825-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 09:32:44 UTC on ccycloud.quasar-gbtldp.root.comops.site
build id 4455440
I20260619 10:17:59.009681 10994 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 10:17:59.009878 10994 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 10:17:59.014626 11002 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 10:17:59.014705 10994 server_base.cc:1034] running on GCE node
W20260619 10:17:59.014640 10999 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 10:17:59.014673 11000 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 10:17:59.015200 10994 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 10:17:59.015441 10994 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 10:17:59.016575 10994 hybrid_clock.cc:648] HybridClock initialized: now 1781864279016566 us; error 27 us; skew 500 ppm
Jun 19 10:17:59 dist-test-slave-6pt2 krb5kdc[10981](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864279, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 10:17:59.019030 10994 init.cc:377] Logged in from keytab as kudu/127.0.206.126@KRBTEST.COM (short username kudu)
I20260619 10:17:59.019425 10994 webserver.cc:460] Webserver started at http://127.0.206.126:46271/ using document root <none> and password file <none>
I20260619 10:17:59.019654 10994 fs_manager.cc:359] Metadata directory not provided
I20260619 10:17:59.019696 10994 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 10:17:59.019853 10994 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 10:17:59.020821 10994 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781864224577058-825-0/minicluster-data/master-0/data/instance:
uuid: "32ffb0e666e2497d9323388ca4461c47"
format_stamp: "Formatted at 2026-06-19 10:17:59 on dist-test-slave-6pt2"
server_key: "a0c7f34e58251dbf5f016b0760a4ffbd"
server_key_iv: "ff6c5b044fd1927161a3790dbf7a54d2"
server_key_version: "encryptionkey@0"
I20260619 10:17:59.021148 10994 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781864224577058-825-0/minicluster-data/master-0/wal/instance:
uuid: "32ffb0e666e2497d9323388ca4461c47"
format_stamp: "Formatted at 2026-06-19 10:17:59 on dist-test-slave-6pt2"
server_key: "a0c7f34e58251dbf5f016b0760a4ffbd"
server_key_iv: "ff6c5b044fd1927161a3790dbf7a54d2"
server_key_version: "encryptionkey@0"
I20260619 10:17:59.022392 10994 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260619 10:17:59.023151 11009 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 10:17:59.023326 10994 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260619 10:17:59.023413 10994 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781864224577058-825-0/minicluster-data/master-0/data,/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781864224577058-825-0/minicluster-data/master-0/wal
uuid: "32ffb0e666e2497d9323388ca4461c47"
format_stamp: "Formatted at 2026-06-19 10:17:59 on dist-test-slave-6pt2"
server_key: "a0c7f34e58251dbf5f016b0760a4ffbd"
server_key_iv: "ff6c5b044fd1927161a3790dbf7a54d2"
server_key_version: "encryptionkey@0"
I20260619 10:17:59.023485 10994 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781864224577058-825-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781864224577058-825-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781864224577058-825-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 10:17:59.068351 10994 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 10:17:59.070178 10994 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 10:17:59.070354 10994 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 10:17:59.074054 10994 rpc_server.cc:307] RPC server started. Bound to: 127.0.206.126:32973
I20260619 10:17:59.074131 11061 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.206.126:32973 every 8 connection(s)
I20260619 10:17:59.074463 10994 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781864224577058-825-0/minicluster-data/master-0/data/info.pb
I20260619 10:17:59.075460 11062 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 10:17:59.077670 11062 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 32ffb0e666e2497d9323388ca4461c47: Bootstrap starting.
I20260619 10:17:59.078246 11062 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 32ffb0e666e2497d9323388ca4461c47: Neither blocks nor log segments found. Creating new log.
I20260619 10:17:59.078542 11062 log.cc:824] T 00000000000000000000000000000000 P 32ffb0e666e2497d9323388ca4461c47: Log is configured to *not* fsync() on all Append() calls
I20260619 10:17:59.079245 11062 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 32ffb0e666e2497d9323388ca4461c47: No bootstrap required, opened a new log
I20260619 10:17:59.080778 11062 raft_consensus.cc:348] T 00000000000000000000000000000000 P 32ffb0e666e2497d9323388ca4461c47 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "32ffb0e666e2497d9323388ca4461c47" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 32973 } }
I20260619 10:17:59.080911 11062 raft_consensus.cc:374] T 00000000000000000000000000000000 P 32ffb0e666e2497d9323388ca4461c47 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 10:17:59.080935 11062 raft_consensus.cc:729] T 00000000000000000000000000000000 P 32ffb0e666e2497d9323388ca4461c47 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 32ffb0e666e2497d9323388ca4461c47, State: Initialized, Role: FOLLOWER
I20260619 10:17:59.081070 11062 consensus_queue.cc:260] T 00000000000000000000000000000000 P 32ffb0e666e2497d9323388ca4461c47 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "32ffb0e666e2497d9323388ca4461c47" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 32973 } }
I20260619 10:17:59.081151 11062 raft_consensus.cc:388] T 00000000000000000000000000000000 P 32ffb0e666e2497d9323388ca4461c47 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 10:17:59.081173 11062 raft_consensus.cc:482] T 00000000000000000000000000000000 P 32ffb0e666e2497d9323388ca4461c47 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 10:17:59.081202 11062 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 32ffb0e666e2497d9323388ca4461c47 [term 0 FOLLOWER]: Advancing to term 1
I20260619 10:17:59.081938 11062 raft_consensus.cc:504] T 00000000000000000000000000000000 P 32ffb0e666e2497d9323388ca4461c47 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "32ffb0e666e2497d9323388ca4461c47" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 32973 } }
I20260619 10:17:59.082022 11062 leader_election.cc:302] T 00000000000000000000000000000000 P 32ffb0e666e2497d9323388ca4461c47 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 32ffb0e666e2497d9323388ca4461c47; no voters:
I20260619 10:17:59.082142 11062 leader_election.cc:288] T 00000000000000000000000000000000 P 32ffb0e666e2497d9323388ca4461c47 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 10:17:59.082212 11065 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 32ffb0e666e2497d9323388ca4461c47 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 10:17:59.082396 11065 raft_consensus.cc:686] T 00000000000000000000000000000000 P 32ffb0e666e2497d9323388ca4461c47 [term 1 LEADER]: Becoming Leader. State: Replica: 32ffb0e666e2497d9323388ca4461c47, State: Running, Role: LEADER
I20260619 10:17:59.082418 11062 sys_catalog.cc:564] T 00000000000000000000000000000000 P 32ffb0e666e2497d9323388ca4461c47 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 10:17:59.082486 11065 consensus_queue.cc:237] T 00000000000000000000000000000000 P 32ffb0e666e2497d9323388ca4461c47 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "32ffb0e666e2497d9323388ca4461c47" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 32973 } }
I20260619 10:17:59.082896 11066 sys_catalog.cc:455] T 00000000000000000000000000000000 P 32ffb0e666e2497d9323388ca4461c47 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "32ffb0e666e2497d9323388ca4461c47" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "32ffb0e666e2497d9323388ca4461c47" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 32973 } } }
I20260619 10:17:59.082909 11067 sys_catalog.cc:455] T 00000000000000000000000000000000 P 32ffb0e666e2497d9323388ca4461c47 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 32ffb0e666e2497d9323388ca4461c47. Latest consensus state: current_term: 1 leader_uuid: "32ffb0e666e2497d9323388ca4461c47" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "32ffb0e666e2497d9323388ca4461c47" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 32973 } } }
I20260619 10:17:59.083110 11067 sys_catalog.cc:458] T 00000000000000000000000000000000 P 32ffb0e666e2497d9323388ca4461c47 [sys.catalog]: This master's current role is: LEADER
I20260619 10:17:59.083338 11066 sys_catalog.cc:458] T 00000000000000000000000000000000 P 32ffb0e666e2497d9323388ca4461c47 [sys.catalog]: This master's current role is: LEADER
I20260619 10:17:59.083457 825 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTU9OP6/build/release/bin/kudu as pid 10994
I20260619 10:17:59.083557 825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781864224577058-825-0/minicluster-data/master-0/wal/instance
I20260619 10:17:59.083777 825 external_mini_cluster.cc:1428] Setting key 8aedd964720f3795752b412d4a8ed597
W20260619 10:17:59.084347 11081 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 32ffb0e666e2497d9323388ca4461c47: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 10:17:59.084394 11081 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 10:17:59.084447 11074 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 10:17:59.084594 11074 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 10:17:59.087018 11074 catalog_manager.cc:1347] Generated new cluster ID: 82e2b88578674fcd9aafaec8543b8615
I20260619 10:17:59.087071 11074 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Jun 19 10:17:59 dist-test-slave-6pt2 krb5kdc[10981](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864278, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.206.126@KRBTEST.COM
I20260619 10:17:59.093650 11084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:59.084404 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51940 (local address 127.0.206.126:32973)
0619 10:17:59.084554 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0619 10:17:59.084565 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0619 10:17:59.084597 (+ 32us) server_negotiation.cc:407] Connection header received
0619 10:17:59.084722 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:17:59.084728 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:17:59.084777 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:17:59.084920 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:17:59.085622 (+ 702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:59.086504 (+ 882us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:59.087492 (+ 988us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:59.087907 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:59.088040 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:59.090348 (+ 2308us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 10:17:59.090352 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 10:17:59.090352 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 10:17:59.090364 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0619 10:17:59.092132 (+ 1768us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:17:59.092497 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:17:59.092499 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:17:59.092500 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:17:59.092605 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:17:59.092819 (+ 214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:17:59.092820 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:17:59.092820 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:17:59.093045 (+ 225us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 10:17:59.093214 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0619 10:17:59.093296 (+ 82us) server_negotiation.cc:299] Negotiation successful
0619 10:17:59.093448 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":103,"thread_start_us":77,"threads_started":1}
I20260619 10:17:59.111979 11074 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 10:17:59.112499 11074 catalog_manager.cc:1504] Loading token signing keys...
I20260619 10:17:59.116456 11074 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 32ffb0e666e2497d9323388ca4461c47: Generated new TSK 0
I20260619 10:17:59.116622 11074 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.0.206.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.206.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.206.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781864224577058-825-0/krb5kdc/kudu_127.0.206.65.keytab.
Entry for principal kudu/127.0.206.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781864224577058-825-0/krb5kdc/kudu_127.0.206.65.keytab.
I20260619 10:17:59.142851 825 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTU9OP6/build/release/bin/kudu
/tmp/dist-test-taskTU9OP6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781864224577058-825-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781864224577058-825-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781864224577058-825-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781864224577058-825-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.206.65:0
--local_ip_for_outbound_sockets=127.0.206.65
--webserver_interface=127.0.206.65
--webserver_port=0
--tserver_master_addrs=127.0.206.126:32973
--keytab_file=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781864224577058-825-0/krb5kdc/kudu_127.0.206.65.keytab
--principal=kudu/127.0.206.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.206.84:36379
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781864224577058-825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781864224577058-825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781864224577058-825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 10:17:59.244598 11088 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 10:17:59.244828 11088 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 10:17:59.244865 11088 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 10:17:59.246009 11088 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 10:17:59.246104 11088 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 10:17:59.246150 11088 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.206.65
I20260619 10:17:59.247680 11088 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.206.84:36379
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781864224577058-825-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781864224577058-825-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781864224577058-825-0/krb5kdc/kudu_127.0.206.65.keytab
--principal=kudu/127.0.206.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.206.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781864224577058-825-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.206.65
--webserver_port=0
--tserver_master_addrs=127.0.206.126:32973
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11088
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.206.65
--log_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781864224577058-825-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 09:32:44 UTC on ccycloud.quasar-gbtldp.root.comops.site
build id 4455440
I20260619 10:17:59.247938 11088 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 10:17:59.248178 11088 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 10:17:59.252972 11094 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 10:17:59.252978 11096 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 10:17:59.252982 11093 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 10:17:59.253278 11088 server_base.cc:1034] running on GCE node
I20260619 10:17:59.253453 11088 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 10:17:59.253628 11088 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 10:17:59.254746 11088 hybrid_clock.cc:648] HybridClock initialized: now 1781864279254728 us; error 31 us; skew 500 ppm
Jun 19 10:17:59 dist-test-slave-6pt2 krb5kdc[10981](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864279, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 10:17:59.257197 11088 init.cc:377] Logged in from keytab as kudu/127.0.206.65@KRBTEST.COM (short username kudu)
I20260619 10:17:59.257689 11088 webserver.cc:460] Webserver started at http://127.0.206.65:42839/ using document root <none> and password file <none>
I20260619 10:17:59.257872 11088 fs_manager.cc:359] Metadata directory not provided
I20260619 10:17:59.257910 11088 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 10:17:59.257992 11088 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 10:17:59.259020 11088 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781864224577058-825-0/minicluster-data/ts-0/data/instance:
uuid: "ba36208dba524eb494d861b98bc13670"
format_stamp: "Formatted at 2026-06-19 10:17:59 on dist-test-slave-6pt2"
server_key: "8b31e1776e17aa1f6c1493413b067b12"
server_key_iv: "428389c8e028b1b868157de04df2b4e3"
server_key_version: "encryptionkey@0"
I20260619 10:17:59.259320 11088 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781864224577058-825-0/minicluster-data/ts-0/wal/instance:
uuid: "ba36208dba524eb494d861b98bc13670"
format_stamp: "Formatted at 2026-06-19 10:17:59 on dist-test-slave-6pt2"
server_key: "8b31e1776e17aa1f6c1493413b067b12"
server_key_iv: "428389c8e028b1b868157de04df2b4e3"
server_key_version: "encryptionkey@0"
I20260619 10:17:59.260505 11088 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 10:17:59.261140 11103 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 10:17:59.261297 11088 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 10:17:59.261391 11088 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781864224577058-825-0/minicluster-data/ts-0/data,/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781864224577058-825-0/minicluster-data/ts-0/wal
uuid: "ba36208dba524eb494d861b98bc13670"
format_stamp: "Formatted at 2026-06-19 10:17:59 on dist-test-slave-6pt2"
server_key: "8b31e1776e17aa1f6c1493413b067b12"
server_key_iv: "428389c8e028b1b868157de04df2b4e3"
server_key_version: "encryptionkey@0"
I20260619 10:17:59.261466 11088 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781864224577058-825-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781864224577058-825-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781864224577058-825-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 10:17:59.272100 11088 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 10:17:59.274209 11088 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 10:17:59.274375 11088 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 10:17:59.274587 11088 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 10:17:59.274958 11088 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 10:17:59.275010 11088 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 10:17:59.275084 11088 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 10:17:59.275112 11088 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 10:17:59.280655 11088 rpc_server.cc:307] RPC server started. Bound to: 127.0.206.65:39921
I20260619 10:17:59.280697 11216 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.206.65:39921 every 8 connection(s)
I20260619 10:17:59.280978 11088 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781864224577058-825-0/minicluster-data/ts-0/data/info.pb
Jun 19 10:17:59 dist-test-slave-6pt2 krb5kdc[10981](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864279, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.65@KRBTEST.COM for kudu/127.0.206.126@KRBTEST.COM
I20260619 10:17:59.287784 825 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTU9OP6/build/release/bin/kudu as pid 11088
I20260619 10:17:59.287886 825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781864224577058-825-0/minicluster-data/ts-0/wal/instance
I20260619 10:17:59.288089 825 external_mini_cluster.cc:1428] Setting key a11bcb5d443d8035463eb96b112c5138
I20260619 10:17:59.289912 11084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:59.281600 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.65:58513 (local address 127.0.206.126:32973)
0619 10:17:59.281670 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0619 10:17:59.281672 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 10:17:59.281832 (+ 160us) server_negotiation.cc:407] Connection header received
0619 10:17:59.282267 (+ 435us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:17:59.282268 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:17:59.282280 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:17:59.282332 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:17:59.283395 (+ 1063us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:59.284018 (+ 623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:59.284976 (+ 958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:59.285157 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:59.285208 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:59.287108 (+ 1900us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 10:17:59.287109 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 10:17:59.287109 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 10:17:59.287116 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 10:17:59.288587 (+ 1471us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:17:59.288990 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:17:59.288991 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:17:59.288991 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:17:59.289048 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:17:59.289295 (+ 247us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:17:59.289296 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:17:59.289297 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:17:59.289405 (+ 108us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 10:17:59.289480 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0619 10:17:59.289740 (+ 260us) server_negotiation.cc:299] Negotiation successful
0619 10:17:59.289820 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260619 10:17:59.290102 11219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:59.281558 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:32973 (local address 127.0.206.65:58513)
0619 10:17:59.281724 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0619 10:17:59.281756 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0619 10:17:59.282190 (+ 434us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 10:17:59.282396 (+ 206us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 10:17:59.282399 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 10:17:59.283029 (+ 630us) client_negotiation.cc:189] Negotiated authn=SASL
0619 10:17:59.283222 (+ 193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:59.283227 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:59.284138 (+ 911us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:59.284139 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:59.284907 (+ 768us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:59.284909 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:59.285225 (+ 316us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:59.285226 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:59.285319 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:59.285728 (+ 409us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 10:17:59.285736 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 10:17:59.287032 (+ 1296us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 10:17:59.288681 (+ 1649us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 10:17:59.288688 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 10:17:59.288691 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 10:17:59.288913 (+ 222us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 10:17:59.289114 (+ 201us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 10:17:59.289115 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 10:17:59.289115 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 10:17:59.289224 (+ 109us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 10:17:59.289472 (+ 248us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 10:17:59.289478 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 10:17:59.289612 (+ 134us) client_negotiation.cc:769] Sending connection context
0619 10:17:59.289718 (+ 106us) client_negotiation.cc:240] Negotiation successful
0619 10:17:59.289876 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":84,"threads_started":1}
I20260619 10:17:59.290535 11217 heartbeater.cc:347] Connected to a master server at 127.0.206.126:32973
I20260619 10:17:59.290664 11217 heartbeater.cc:464] Registering TS with master...
I20260619 10:17:59.290843 11217 heartbeater.cc:511] Master 127.0.206.126:32973 requested a full tablet report, sending...
I20260619 10:17:59.291319 11026 ts_manager.cc:194] Registered new tserver with Master: ba36208dba524eb494d861b98bc13670 (127.0.206.65:39921)
I20260619 10:17:59.291877 11026 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.206.65@KRBTEST.COM'} at 127.0.206.65:58513
WARNING: no policy specified for kudu/127.0.206.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.206.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.206.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781864224577058-825-0/krb5kdc/kudu_127.0.206.66.keytab.
Entry for principal kudu/127.0.206.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781864224577058-825-0/krb5kdc/kudu_127.0.206.66.keytab.
I20260619 10:17:59.311435 825 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTU9OP6/build/release/bin/kudu
/tmp/dist-test-taskTU9OP6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781864224577058-825-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781864224577058-825-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781864224577058-825-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781864224577058-825-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.206.66:0
--local_ip_for_outbound_sockets=127.0.206.66
--webserver_interface=127.0.206.66
--webserver_port=0
--tserver_master_addrs=127.0.206.126:32973
--keytab_file=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781864224577058-825-0/krb5kdc/kudu_127.0.206.66.keytab
--principal=kudu/127.0.206.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.206.84:36379
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781864224577058-825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781864224577058-825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781864224577058-825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 10:17:59.389041 11222 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 10:17:59.389256 11222 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 10:17:59.389293 11222 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 10:17:59.390465 11222 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 10:17:59.390520 11222 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 10:17:59.390561 11222 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.206.66
I20260619 10:17:59.391996 11222 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.206.84:36379
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781864224577058-825-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781864224577058-825-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781864224577058-825-0/krb5kdc/kudu_127.0.206.66.keytab
--principal=kudu/127.0.206.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.206.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781864224577058-825-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.206.66
--webserver_port=0
--tserver_master_addrs=127.0.206.126:32973
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11222
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.206.66
--log_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781864224577058-825-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 09:32:44 UTC on ccycloud.quasar-gbtldp.root.comops.site
build id 4455440
I20260619 10:17:59.392258 11222 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 10:17:59.392470 11222 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 10:17:59.397315 11227 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 10:17:59.397307 11230 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 10:17:59.397465 11222 server_base.cc:1034] running on GCE node
W20260619 10:17:59.397308 11228 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 10:17:59.397727 11222 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 10:17:59.397971 11222 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 10:17:59.399108 11222 hybrid_clock.cc:648] HybridClock initialized: now 1781864279399104 us; error 31 us; skew 500 ppm
Jun 19 10:17:59 dist-test-slave-6pt2 krb5kdc[10981](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864279, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 10:17:59.401481 11222 init.cc:377] Logged in from keytab as kudu/127.0.206.66@KRBTEST.COM (short username kudu)
I20260619 10:17:59.401934 11222 webserver.cc:460] Webserver started at http://127.0.206.66:36779/ using document root <none> and password file <none>
I20260619 10:17:59.402134 11222 fs_manager.cc:359] Metadata directory not provided
I20260619 10:17:59.402197 11222 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 10:17:59.402345 11222 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 10:17:59.403398 11222 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781864224577058-825-0/minicluster-data/ts-1/data/instance:
uuid: "94f6ba3de346413088653c90bad52514"
format_stamp: "Formatted at 2026-06-19 10:17:59 on dist-test-slave-6pt2"
server_key: "dbe1950a5de1347abf4e80ae4ce6c74d"
server_key_iv: "1c43ecdd797b6061f86512781148fbf1"
server_key_version: "encryptionkey@0"
I20260619 10:17:59.403745 11222 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781864224577058-825-0/minicluster-data/ts-1/wal/instance:
uuid: "94f6ba3de346413088653c90bad52514"
format_stamp: "Formatted at 2026-06-19 10:17:59 on dist-test-slave-6pt2"
server_key: "dbe1950a5de1347abf4e80ae4ce6c74d"
server_key_iv: "1c43ecdd797b6061f86512781148fbf1"
server_key_version: "encryptionkey@0"
I20260619 10:17:59.404986 11222 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260619 10:17:59.405658 11237 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 10:17:59.405819 11222 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260619 10:17:59.405907 11222 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781864224577058-825-0/minicluster-data/ts-1/data,/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781864224577058-825-0/minicluster-data/ts-1/wal
uuid: "94f6ba3de346413088653c90bad52514"
format_stamp: "Formatted at 2026-06-19 10:17:59 on dist-test-slave-6pt2"
server_key: "dbe1950a5de1347abf4e80ae4ce6c74d"
server_key_iv: "1c43ecdd797b6061f86512781148fbf1"
server_key_version: "encryptionkey@0"
I20260619 10:17:59.405988 11222 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781864224577058-825-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781864224577058-825-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781864224577058-825-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 10:17:59.416100 11222 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 10:17:59.417721 11222 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 10:17:59.417861 11222 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 10:17:59.418051 11222 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 10:17:59.418330 11222 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 10:17:59.418393 11222 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 10:17:59.418426 11222 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 10:17:59.418450 11222 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 10:17:59.424440 11222 rpc_server.cc:307] RPC server started. Bound to: 127.0.206.66:36095
I20260619 10:17:59.424484 11350 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.206.66:36095 every 8 connection(s)
I20260619 10:17:59.424743 11222 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781864224577058-825-0/minicluster-data/ts-1/data/info.pb
I20260619 10:17:59.425792 825 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTU9OP6/build/release/bin/kudu as pid 11222
I20260619 10:17:59.425891 825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781864224577058-825-0/minicluster-data/ts-1/wal/instance
I20260619 10:17:59.426103 825 external_mini_cluster.cc:1428] Setting key f1cbbf2077cb1e509564aa8466cced67
Jun 19 10:17:59 dist-test-slave-6pt2 krb5kdc[10981](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864279, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.66@KRBTEST.COM for kudu/127.0.206.126@KRBTEST.COM
I20260619 10:17:59.434671 11084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:59.425353 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.66:51757 (local address 127.0.206.126:32973)
0619 10:17:59.425458 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0619 10:17:59.425460 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 10:17:59.425679 (+ 219us) server_negotiation.cc:407] Connection header received
0619 10:17:59.426370 (+ 691us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:17:59.426372 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:17:59.426384 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:17:59.426441 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:17:59.427730 (+ 1289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:59.428211 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:59.429228 (+ 1017us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:59.429538 (+ 310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:59.429608 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:59.431713 (+ 2105us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 10:17:59.431714 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 10:17:59.431715 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 10:17:59.431723 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 10:17:59.433283 (+ 1560us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:17:59.433686 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:17:59.433687 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:17:59.433688 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:17:59.433744 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:17:59.433973 (+ 229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:17:59.433975 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:17:59.433975 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:17:59.434112 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 10:17:59.434183 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0619 10:17:59.434451 (+ 268us) server_negotiation.cc:299] Negotiation successful
0619 10:17:59.434553 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260619 10:17:59.434724 11353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:59.425358 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:32973 (local address 127.0.206.66:51757)
0619 10:17:59.425536 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0619 10:17:59.425567 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0619 10:17:59.426254 (+ 687us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 10:17:59.426521 (+ 267us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 10:17:59.426524 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 10:17:59.427343 (+ 819us) client_negotiation.cc:189] Negotiated authn=SASL
0619 10:17:59.427640 (+ 297us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:59.427646 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:59.428332 (+ 686us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:59.428333 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:59.429113 (+ 780us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:59.429116 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:59.429621 (+ 505us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:59.429622 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:59.429724 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:59.430223 (+ 499us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 10:17:59.430231 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 10:17:59.431618 (+ 1387us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 10:17:59.433378 (+ 1760us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 10:17:59.433383 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 10:17:59.433385 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 10:17:59.433603 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 10:17:59.433823 (+ 220us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 10:17:59.433824 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 10:17:59.433824 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 10:17:59.433909 (+ 85us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 10:17:59.434184 (+ 275us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 10:17:59.434187 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 10:17:59.434293 (+ 106us) client_negotiation.cc:769] Sending connection context
0619 10:17:59.434385 (+ 92us) client_negotiation.cc:240] Negotiation successful
0619 10:17:59.434523 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":105,"threads_started":1}
WARNING: no policy specified for kudu/127.0.206.67@KRBTEST.COM; defaulting to no policy
I20260619 10:17:59.435413 11351 heartbeater.cc:347] Connected to a master server at 127.0.206.126:32973
I20260619 10:17:59.435488 11351 heartbeater.cc:464] Registering TS with master...
I20260619 10:17:59.435612 11351 heartbeater.cc:511] Master 127.0.206.126:32973 requested a full tablet report, sending...
I20260619 10:17:59.435973 11026 ts_manager.cc:194] Registered new tserver with Master: 94f6ba3de346413088653c90bad52514 (127.0.206.66:36095)
I20260619 10:17:59.436342 11026 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.206.66@KRBTEST.COM'} at 127.0.206.66:51757
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.206.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.206.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781864224577058-825-0/krb5kdc/kudu_127.0.206.67.keytab.
Entry for principal kudu/127.0.206.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781864224577058-825-0/krb5kdc/kudu_127.0.206.67.keytab.
I20260619 10:17:59.450410 825 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTU9OP6/build/release/bin/kudu
/tmp/dist-test-taskTU9OP6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781864224577058-825-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781864224577058-825-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781864224577058-825-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781864224577058-825-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.206.67:0
--local_ip_for_outbound_sockets=127.0.206.67
--webserver_interface=127.0.206.67
--webserver_port=0
--tserver_master_addrs=127.0.206.126:32973
--keytab_file=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781864224577058-825-0/krb5kdc/kudu_127.0.206.67.keytab
--principal=kudu/127.0.206.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.206.84:36379
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781864224577058-825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781864224577058-825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781864224577058-825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 10:17:59.527359 11356 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 10:17:59.527550 11356 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 10:17:59.527573 11356 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 10:17:59.528693 11356 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 10:17:59.528733 11356 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 10:17:59.528781 11356 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.206.67
I20260619 10:17:59.530145 11356 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.206.84:36379
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781864224577058-825-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781864224577058-825-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781864224577058-825-0/krb5kdc/kudu_127.0.206.67.keytab
--principal=kudu/127.0.206.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.206.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781864224577058-825-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.206.67
--webserver_port=0
--tserver_master_addrs=127.0.206.126:32973
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11356
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.206.67
--log_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781864224577058-825-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 09:32:44 UTC on ccycloud.quasar-gbtldp.root.comops.site
build id 4455440
I20260619 10:17:59.530390 11356 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 10:17:59.530614 11356 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 10:17:59.535286 11364 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 10:17:59.535300 11361 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 10:17:59.535322 11362 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 10:17:59.535535 11356 server_base.cc:1034] running on GCE node
I20260619 10:17:59.535722 11356 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 10:17:59.535959 11356 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 10:17:59.537118 11356 hybrid_clock.cc:648] HybridClock initialized: now 1781864279537091 us; error 53 us; skew 500 ppm
Jun 19 10:17:59 dist-test-slave-6pt2 krb5kdc[10981](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864279, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 10:17:59.539480 11356 init.cc:377] Logged in from keytab as kudu/127.0.206.67@KRBTEST.COM (short username kudu)
I20260619 10:17:59.539912 11356 webserver.cc:460] Webserver started at http://127.0.206.67:39395/ using document root <none> and password file <none>
I20260619 10:17:59.540124 11356 fs_manager.cc:359] Metadata directory not provided
I20260619 10:17:59.540186 11356 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 10:17:59.540329 11356 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 10:17:59.541358 11356 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781864224577058-825-0/minicluster-data/ts-2/data/instance:
uuid: "f17a786d859d4913bdefb01bdd7323d3"
format_stamp: "Formatted at 2026-06-19 10:17:59 on dist-test-slave-6pt2"
server_key: "cbe211ae9545f0c35254a277df8a19ec"
server_key_iv: "2ccd2398bc34f5634db5680cb4cfc174"
server_key_version: "encryptionkey@0"
I20260619 10:17:59.541697 11356 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781864224577058-825-0/minicluster-data/ts-2/wal/instance:
uuid: "f17a786d859d4913bdefb01bdd7323d3"
format_stamp: "Formatted at 2026-06-19 10:17:59 on dist-test-slave-6pt2"
server_key: "cbe211ae9545f0c35254a277df8a19ec"
server_key_iv: "2ccd2398bc34f5634db5680cb4cfc174"
server_key_version: "encryptionkey@0"
I20260619 10:17:59.542979 11356 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.002s
I20260619 10:17:59.543677 11371 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 10:17:59.543853 11356 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 10:17:59.543948 11356 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781864224577058-825-0/minicluster-data/ts-2/data,/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781864224577058-825-0/minicluster-data/ts-2/wal
uuid: "f17a786d859d4913bdefb01bdd7323d3"
format_stamp: "Formatted at 2026-06-19 10:17:59 on dist-test-slave-6pt2"
server_key: "cbe211ae9545f0c35254a277df8a19ec"
server_key_iv: "2ccd2398bc34f5634db5680cb4cfc174"
server_key_version: "encryptionkey@0"
I20260619 10:17:59.544021 11356 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781864224577058-825-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781864224577058-825-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781864224577058-825-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 10:17:59.563063 11356 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 10:17:59.564939 11356 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 10:17:59.565084 11356 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 10:17:59.565340 11356 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 10:17:59.565632 11356 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 10:17:59.565683 11356 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 10:17:59.565724 11356 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 10:17:59.565749 11356 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 10:17:59.571358 11356 rpc_server.cc:307] RPC server started. Bound to: 127.0.206.67:38701
I20260619 10:17:59.571455 11484 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.206.67:38701 every 8 connection(s)
I20260619 10:17:59.571699 11356 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781864224577058-825-0/minicluster-data/ts-2/data/info.pb
I20260619 10:17:59.575057 825 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTU9OP6/build/release/bin/kudu as pid 11356
I20260619 10:17:59.575155 825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781864224577058-825-0/minicluster-data/ts-2/wal/instance
I20260619 10:17:59.575361 825 external_mini_cluster.cc:1428] Setting key e1c83b84bf6fdae9787e885df5a033c6
Jun 19 10:17:59 dist-test-slave-6pt2 krb5kdc[10981](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864279, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.67@KRBTEST.COM for kudu/127.0.206.126@KRBTEST.COM
I20260619 10:17:59.580663 11084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:59.572367 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.67:55845 (local address 127.0.206.126:32973)
0619 10:17:59.572423 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0619 10:17:59.572425 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 10:17:59.572610 (+ 185us) server_negotiation.cc:407] Connection header received
0619 10:17:59.573045 (+ 435us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:17:59.573046 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:17:59.573055 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:17:59.573107 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:17:59.573980 (+ 873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:59.574466 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:59.575798 (+ 1332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:59.576100 (+ 302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:59.576167 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:59.578133 (+ 1966us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 10:17:59.578134 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 10:17:59.578135 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 10:17:59.578141 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 10:17:59.579574 (+ 1433us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:17:59.579936 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:17:59.579937 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:17:59.579937 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:17:59.579986 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:17:59.580181 (+ 195us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:17:59.580181 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:17:59.580182 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:17:59.580281 (+ 99us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 10:17:59.580330 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0619 10:17:59.580537 (+ 207us) server_negotiation.cc:299] Negotiation successful
0619 10:17:59.580592 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260619 10:17:59.580812 11487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:59.572341 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:32973 (local address 127.0.206.67:55845)
0619 10:17:59.572488 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0619 10:17:59.572526 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0619 10:17:59.572970 (+ 444us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 10:17:59.573154 (+ 184us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 10:17:59.573157 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 10:17:59.573707 (+ 550us) client_negotiation.cc:189] Negotiated authn=SASL
0619 10:17:59.573895 (+ 188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:59.573900 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:59.574540 (+ 640us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:59.574541 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:59.575691 (+ 1150us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:59.575693 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:59.576197 (+ 504us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:59.576198 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:59.576351 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:59.576758 (+ 407us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 10:17:59.576766 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 10:17:59.578055 (+ 1289us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 10:17:59.579647 (+ 1592us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 10:17:59.579652 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 10:17:59.579653 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 10:17:59.579872 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 10:17:59.580042 (+ 170us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 10:17:59.580043 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 10:17:59.580043 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 10:17:59.580130 (+ 87us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 10:17:59.580334 (+ 204us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 10:17:59.580337 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 10:17:59.580435 (+ 98us) client_negotiation.cc:769] Sending connection context
0619 10:17:59.580511 (+ 76us) client_negotiation.cc:240] Negotiation successful
0619 10:17:59.580629 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":82,"threads_started":1}
I20260619 10:17:59.581231 11485 heartbeater.cc:347] Connected to a master server at 127.0.206.126:32973
I20260619 10:17:59.581302 11485 heartbeater.cc:464] Registering TS with master...
I20260619 10:17:59.581481 11485 heartbeater.cc:511] Master 127.0.206.126:32973 requested a full tablet report, sending...
I20260619 10:17:59.581813 11026 ts_manager.cc:194] Registered new tserver with Master: f17a786d859d4913bdefb01bdd7323d3 (127.0.206.67:38701)
I20260619 10:17:59.582224 11026 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.206.67@KRBTEST.COM'} at 127.0.206.67:55845
I20260619 10:17:59.586939 825 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 10:17:59.593333 11084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:59.588170 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51948 (local address 127.0.206.126:32973)
0619 10:17:59.588236 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0619 10:17:59.588238 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 10:17:59.588280 (+ 42us) server_negotiation.cc:407] Connection header received
0619 10:17:59.588330 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:17:59.588330 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:17:59.588338 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:17:59.588387 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:17:59.588892 (+ 505us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:59.589329 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:59.590003 (+ 674us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:59.590168 (+ 165us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:59.590219 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:59.591328 (+ 1109us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 10:17:59.591329 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 10:17:59.591330 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 10:17:59.591335 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0619 10:17:59.592488 (+ 1153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:17:59.592780 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:17:59.592781 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:17:59.592781 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:17:59.592824 (+ 43us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:17:59.592951 (+ 127us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:17:59.592951 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:17:59.592951 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:17:59.593064 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 10:17:59.593133 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0619 10:17:59.593224 (+ 91us) server_negotiation.cc:299] Negotiation successful
0619 10:17:59.593275 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260619 10:17:59.594991 11026 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51948:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 10:17:59.596474 11026 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 10:17:59.602703 11494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:59.598475 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.66:36095 (local address 127.0.0.1:58676)
0619 10:17:59.598640 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0619 10:17:59.598683 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0619 10:17:59.598749 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 10:17:59.599236 (+ 487us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 10:17:59.599238 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 10:17:59.599240 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 10:17:59.599303 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:59.599305 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:59.600293 (+ 988us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:59.600294 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:59.601453 (+ 1159us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:59.601455 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:59.602358 (+ 903us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:59.602360 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:59.602436 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:59.602449 (+ 13us) client_negotiation.cc:769] Sending connection context
0619 10:17:59.602530 (+ 81us) client_negotiation.cc:240] Negotiation successful
0619 10:17:59.602580 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":81,"threads_started":1}
I20260619 10:17:59.602816 11493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:59.598524 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.65:39921 (local address 127.0.0.1:42386)
0619 10:17:59.598808 (+ 284us) negotiation.cc:107] Waiting for socket to connect
0619 10:17:59.598818 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 10:17:59.598868 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 10:17:59.599235 (+ 367us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 10:17:59.599237 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 10:17:59.599239 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 10:17:59.599324 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:59.599327 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:59.600695 (+ 1368us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:59.600696 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:59.601725 (+ 1029us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:59.601726 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:59.602550 (+ 824us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:59.602551 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:59.602635 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:59.602639 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 10:17:59.602703 (+ 64us) client_negotiation.cc:240] Negotiation successful
0619 10:17:59.602744 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":272,"thread_start_us":54,"threads_started":1}
I20260619 10:17:59.603062 11496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:59.598718 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.67:38701 (local address 127.0.0.1:37968)
0619 10:17:59.599306 (+ 588us) negotiation.cc:107] Waiting for socket to connect
0619 10:17:59.599316 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 10:17:59.599368 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 10:17:59.599580 (+ 212us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 10:17:59.599581 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 10:17:59.599583 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 10:17:59.599622 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:59.599624 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:59.600818 (+ 1194us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:59.600819 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:59.601840 (+ 1021us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:59.601842 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:59.602699 (+ 857us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:59.602700 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:59.602944 (+ 244us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:59.602949 (+ 5us) client_negotiation.cc:769] Sending connection context
0619 10:17:59.602992 (+ 43us) client_negotiation.cc:240] Negotiation successful
0619 10:17:59.603021 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":575,"thread_start_us":43,"threads_started":1}
I20260619 10:17:59.603098 11495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:59.598681 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58676 (local address 127.0.206.66:36095)
0619 10:17:59.599082 (+ 401us) server_negotiation.cc:206] Beginning negotiation
0619 10:17:59.599086 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 10:17:59.599095 (+ 9us) server_negotiation.cc:407] Connection header received
0619 10:17:59.599130 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:17:59.599132 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:17:59.599148 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:17:59.599219 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 10:17:59.599388 (+ 169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:59.600189 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:59.601543 (+ 1354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:59.602274 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:59.602346 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:59.602398 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0619 10:17:59.602981 (+ 583us) server_negotiation.cc:299] Negotiation successful
0619 10:17:59.603026 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":371,"thread_start_us":75,"threads_started":1}
I20260619 10:17:59.603108 11497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:59.598806 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37968 (local address 127.0.206.67:38701)
0619 10:17:59.598961 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0619 10:17:59.598966 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 10:17:59.599429 (+ 463us) server_negotiation.cc:407] Connection header received
0619 10:17:59.599473 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:17:59.599475 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:17:59.599489 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:17:59.599569 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 10:17:59.599704 (+ 135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:59.600706 (+ 1002us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:59.601922 (+ 1216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:59.602631 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:59.602701 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:59.602751 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0619 10:17:59.603006 (+ 255us) server_negotiation.cc:299] Negotiation successful
0619 10:17:59.603042 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":125,"thread_start_us":97,"threads_started":1}
I20260619 10:17:59.603359 11498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:59.598883 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42386 (local address 127.0.206.65:39921)
0619 10:17:59.599023 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0619 10:17:59.599028 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 10:17:59.599038 (+ 10us) server_negotiation.cc:407] Connection header received
0619 10:17:59.599079 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:17:59.599081 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:17:59.599098 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:17:59.599404 (+ 306us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 10:17:59.599475 (+ 71us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:59.600589 (+ 1114us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:59.601806 (+ 1217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:59.602469 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:59.602537 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:59.602587 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0619 10:17:59.603180 (+ 593us) server_negotiation.cc:299] Negotiation successful
0619 10:17:59.603239 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":105,"thread_start_us":72,"threads_started":1}
I20260619 10:17:59.603766 11285 tablet_service.cc:1467] Processing CreateTablet for tablet 52a667864bb2468e946bac9e732ed721 (DEFAULT_TABLE table=test-table [id=fda22cca04d249e0a9705b7fec993a88]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 10:17:59.603885 11151 tablet_service.cc:1467] Processing CreateTablet for tablet 52a667864bb2468e946bac9e732ed721 (DEFAULT_TABLE table=test-table [id=fda22cca04d249e0a9705b7fec993a88]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 10:17:59.603886 11419 tablet_service.cc:1467] Processing CreateTablet for tablet 52a667864bb2468e946bac9e732ed721 (DEFAULT_TABLE table=test-table [id=fda22cca04d249e0a9705b7fec993a88]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 10:17:59.604017 11285 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 52a667864bb2468e946bac9e732ed721. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 10:17:59.604176 11151 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 52a667864bb2468e946bac9e732ed721. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 10:17:59.604176 11419 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 52a667864bb2468e946bac9e732ed721. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 10:17:59.606591 11499 tablet_bootstrap.cc:492] T 52a667864bb2468e946bac9e732ed721 P 94f6ba3de346413088653c90bad52514: Bootstrap starting.
I20260619 10:17:59.607113 11499 tablet_bootstrap.cc:654] T 52a667864bb2468e946bac9e732ed721 P 94f6ba3de346413088653c90bad52514: Neither blocks nor log segments found. Creating new log.
I20260619 10:17:59.607307 11501 tablet_bootstrap.cc:492] T 52a667864bb2468e946bac9e732ed721 P f17a786d859d4913bdefb01bdd7323d3: Bootstrap starting.
I20260619 10:17:59.607666 11499 log.cc:824] T 52a667864bb2468e946bac9e732ed721 P 94f6ba3de346413088653c90bad52514: Log is configured to *not* fsync() on all Append() calls
I20260619 10:17:59.607949 11501 tablet_bootstrap.cc:654] T 52a667864bb2468e946bac9e732ed721 P f17a786d859d4913bdefb01bdd7323d3: Neither blocks nor log segments found. Creating new log.
I20260619 10:17:59.608462 11501 log.cc:824] T 52a667864bb2468e946bac9e732ed721 P f17a786d859d4913bdefb01bdd7323d3: Log is configured to *not* fsync() on all Append() calls
I20260619 10:17:59.608487 11499 tablet_bootstrap.cc:492] T 52a667864bb2468e946bac9e732ed721 P 94f6ba3de346413088653c90bad52514: No bootstrap required, opened a new log
I20260619 10:17:59.608546 11499 ts_tablet_manager.cc:1397] T 52a667864bb2468e946bac9e732ed721 P 94f6ba3de346413088653c90bad52514: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260619 10:17:59.608783 11500 tablet_bootstrap.cc:492] T 52a667864bb2468e946bac9e732ed721 P ba36208dba524eb494d861b98bc13670: Bootstrap starting.
I20260619 10:17:59.609149 11501 tablet_bootstrap.cc:492] T 52a667864bb2468e946bac9e732ed721 P f17a786d859d4913bdefb01bdd7323d3: No bootstrap required, opened a new log
I20260619 10:17:59.609243 11501 ts_tablet_manager.cc:1397] T 52a667864bb2468e946bac9e732ed721 P f17a786d859d4913bdefb01bdd7323d3: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260619 10:17:59.609416 11500 tablet_bootstrap.cc:654] T 52a667864bb2468e946bac9e732ed721 P ba36208dba524eb494d861b98bc13670: Neither blocks nor log segments found. Creating new log.
I20260619 10:17:59.609966 11500 log.cc:824] T 52a667864bb2468e946bac9e732ed721 P ba36208dba524eb494d861b98bc13670: Log is configured to *not* fsync() on all Append() calls
I20260619 10:17:59.610780 11500 tablet_bootstrap.cc:492] T 52a667864bb2468e946bac9e732ed721 P ba36208dba524eb494d861b98bc13670: No bootstrap required, opened a new log
I20260619 10:17:59.610662 11499 raft_consensus.cc:348] T 52a667864bb2468e946bac9e732ed721 P 94f6ba3de346413088653c90bad52514 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94f6ba3de346413088653c90bad52514" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 36095 } } peers { permanent_uuid: "f17a786d859d4913bdefb01bdd7323d3" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 38701 } } peers { permanent_uuid: "ba36208dba524eb494d861b98bc13670" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 39921 } }
I20260619 10:17:59.610846 11500 ts_tablet_manager.cc:1397] T 52a667864bb2468e946bac9e732ed721 P ba36208dba524eb494d861b98bc13670: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260619 10:17:59.610845 11499 raft_consensus.cc:374] T 52a667864bb2468e946bac9e732ed721 P 94f6ba3de346413088653c90bad52514 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 10:17:59.610881 11499 raft_consensus.cc:729] T 52a667864bb2468e946bac9e732ed721 P 94f6ba3de346413088653c90bad52514 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 94f6ba3de346413088653c90bad52514, State: Initialized, Role: FOLLOWER
I20260619 10:17:59.611016 11499 consensus_queue.cc:260] T 52a667864bb2468e946bac9e732ed721 P 94f6ba3de346413088653c90bad52514 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94f6ba3de346413088653c90bad52514" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 36095 } } peers { permanent_uuid: "f17a786d859d4913bdefb01bdd7323d3" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 38701 } } peers { permanent_uuid: "ba36208dba524eb494d861b98bc13670" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 39921 } }
I20260619 10:17:59.611272 11499 ts_tablet_manager.cc:1428] T 52a667864bb2468e946bac9e732ed721 P 94f6ba3de346413088653c90bad52514: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20260619 10:17:59.611394 11351 heartbeater.cc:503] Master 127.0.206.126:32973 was elected leader, sending a full tablet report...
I20260619 10:17:59.611322 11501 raft_consensus.cc:348] T 52a667864bb2468e946bac9e732ed721 P f17a786d859d4913bdefb01bdd7323d3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94f6ba3de346413088653c90bad52514" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 36095 } } peers { permanent_uuid: "f17a786d859d4913bdefb01bdd7323d3" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 38701 } } peers { permanent_uuid: "ba36208dba524eb494d861b98bc13670" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 39921 } }
I20260619 10:17:59.611788 11501 raft_consensus.cc:374] T 52a667864bb2468e946bac9e732ed721 P f17a786d859d4913bdefb01bdd7323d3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 10:17:59.611833 11501 raft_consensus.cc:729] T 52a667864bb2468e946bac9e732ed721 P f17a786d859d4913bdefb01bdd7323d3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f17a786d859d4913bdefb01bdd7323d3, State: Initialized, Role: FOLLOWER
I20260619 10:17:59.611950 11501 consensus_queue.cc:260] T 52a667864bb2468e946bac9e732ed721 P f17a786d859d4913bdefb01bdd7323d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94f6ba3de346413088653c90bad52514" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 36095 } } peers { permanent_uuid: "f17a786d859d4913bdefb01bdd7323d3" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 38701 } } peers { permanent_uuid: "ba36208dba524eb494d861b98bc13670" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 39921 } }
I20260619 10:17:59.612288 11501 ts_tablet_manager.cc:1428] T 52a667864bb2468e946bac9e732ed721 P f17a786d859d4913bdefb01bdd7323d3: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260619 10:17:59.612406 11485 heartbeater.cc:503] Master 127.0.206.126:32973 was elected leader, sending a full tablet report...
I20260619 10:17:59.612785 11500 raft_consensus.cc:348] T 52a667864bb2468e946bac9e732ed721 P ba36208dba524eb494d861b98bc13670 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94f6ba3de346413088653c90bad52514" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 36095 } } peers { permanent_uuid: "f17a786d859d4913bdefb01bdd7323d3" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 38701 } } peers { permanent_uuid: "ba36208dba524eb494d861b98bc13670" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 39921 } }
I20260619 10:17:59.612917 11500 raft_consensus.cc:374] T 52a667864bb2468e946bac9e732ed721 P ba36208dba524eb494d861b98bc13670 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 10:17:59.612957 11500 raft_consensus.cc:729] T 52a667864bb2468e946bac9e732ed721 P ba36208dba524eb494d861b98bc13670 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ba36208dba524eb494d861b98bc13670, State: Initialized, Role: FOLLOWER
I20260619 10:17:59.613063 11500 consensus_queue.cc:260] T 52a667864bb2468e946bac9e732ed721 P ba36208dba524eb494d861b98bc13670 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94f6ba3de346413088653c90bad52514" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 36095 } } peers { permanent_uuid: "f17a786d859d4913bdefb01bdd7323d3" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 38701 } } peers { permanent_uuid: "ba36208dba524eb494d861b98bc13670" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 39921 } }
I20260619 10:17:59.613317 11500 ts_tablet_manager.cc:1428] T 52a667864bb2468e946bac9e732ed721 P ba36208dba524eb494d861b98bc13670: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260619 10:17:59.613324 11217 heartbeater.cc:503] Master 127.0.206.126:32973 was elected leader, sending a full tablet report...
I20260619 10:17:59.653404 11505 raft_consensus.cc:482] T 52a667864bb2468e946bac9e732ed721 P 94f6ba3de346413088653c90bad52514 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 10:17:59.653530 11505 raft_consensus.cc:504] T 52a667864bb2468e946bac9e732ed721 P 94f6ba3de346413088653c90bad52514 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94f6ba3de346413088653c90bad52514" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 36095 } } peers { permanent_uuid: "f17a786d859d4913bdefb01bdd7323d3" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 38701 } } peers { permanent_uuid: "ba36208dba524eb494d861b98bc13670" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 39921 } }
I20260619 10:17:59.653878 11505 leader_election.cc:288] T 52a667864bb2468e946bac9e732ed721 P 94f6ba3de346413088653c90bad52514 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f17a786d859d4913bdefb01bdd7323d3 (127.0.206.67:38701), ba36208dba524eb494d861b98bc13670 (127.0.206.65:39921)
I20260619 10:17:59.657734 11498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:59.654141 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.66:55569 (local address 127.0.206.65:39921)
0619 10:17:59.654186 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0619 10:17:59.654188 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 10:17:59.654329 (+ 141us) server_negotiation.cc:407] Connection header received
0619 10:17:59.654355 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:17:59.654356 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:17:59.654365 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:17:59.654417 (+ 52us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 10:17:59.654546 (+ 129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:59.655335 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:59.656816 (+ 1481us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:59.657395 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:59.657472 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:59.657480 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0619 10:17:59.657631 (+ 151us) server_negotiation.cc:299] Negotiation successful
0619 10:17:59.657666 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260619 10:17:59.657734 11497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:59.654126 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.66:57197 (local address 127.0.206.67:38701)
0619 10:17:59.654171 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0619 10:17:59.654174 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 10:17:59.654182 (+ 8us) server_negotiation.cc:407] Connection header received
0619 10:17:59.654286 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:17:59.654287 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:17:59.654295 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:17:59.654376 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 10:17:59.654548 (+ 172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:59.655337 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:59.656816 (+ 1479us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:59.657396 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:59.657472 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:59.657480 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0619 10:17:59.657631 (+ 151us) server_negotiation.cc:299] Negotiation successful
0619 10:17:59.657668 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260619 10:17:59.657747 11353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:59.654038 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.67:38701 (local address 127.0.206.66:57197)
0619 10:17:59.654127 (+ 89us) negotiation.cc:107] Waiting for socket to connect
0619 10:17:59.654139 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 10:17:59.654195 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 10:17:59.654388 (+ 193us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 10:17:59.654389 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 10:17:59.654390 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 10:17:59.654473 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:59.654475 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:59.655420 (+ 945us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:59.655421 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:59.656726 (+ 1305us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:59.656729 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:59.657477 (+ 748us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:59.657478 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:59.657556 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:59.657558 (+ 2us) client_negotiation.cc:769] Sending connection context
0619 10:17:59.657614 (+ 56us) client_negotiation.cc:240] Negotiation successful
0619 10:17:59.657655 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":72,"mutex_wait_us":72}
I20260619 10:17:59.657753 11508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:59.654077 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.65:39921 (local address 127.0.206.66:55569)
0619 10:17:59.654255 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0619 10:17:59.654263 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0619 10:17:59.654313 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 10:17:59.654423 (+ 110us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 10:17:59.654423 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 10:17:59.654425 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 10:17:59.654477 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:59.654478 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:59.655418 (+ 940us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:59.655419 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:59.656729 (+ 1310us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:17:59.656731 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:17:59.657478 (+ 747us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:17:59.657479 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:17:59.657552 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:59.657556 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 10:17:59.657614 (+ 58us) client_negotiation.cc:240] Negotiation successful
0619 10:17:59.657655 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":161,"mutex_wait_us":39,"thread_start_us":130,"threads_started":1}
I20260619 10:17:59.658072 11171 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "52a667864bb2468e946bac9e732ed721" candidate_uuid: "94f6ba3de346413088653c90bad52514" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba36208dba524eb494d861b98bc13670" is_pre_election: true
I20260619 10:17:59.658201 11171 raft_consensus.cc:2454] T 52a667864bb2468e946bac9e732ed721 P ba36208dba524eb494d861b98bc13670 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94f6ba3de346413088653c90bad52514 in term 0.
I20260619 10:17:59.658191 11439 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "52a667864bb2468e946bac9e732ed721" candidate_uuid: "94f6ba3de346413088653c90bad52514" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f17a786d859d4913bdefb01bdd7323d3" is_pre_election: true
I20260619 10:17:59.658336 11439 raft_consensus.cc:2454] T 52a667864bb2468e946bac9e732ed721 P f17a786d859d4913bdefb01bdd7323d3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94f6ba3de346413088653c90bad52514 in term 0.
I20260619 10:17:59.658452 11238 leader_election.cc:302] T 52a667864bb2468e946bac9e732ed721 P 94f6ba3de346413088653c90bad52514 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 94f6ba3de346413088653c90bad52514, ba36208dba524eb494d861b98bc13670; no voters:
I20260619 10:17:59.658615 11505 raft_consensus.cc:2781] T 52a667864bb2468e946bac9e732ed721 P 94f6ba3de346413088653c90bad52514 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 10:17:59.658679 11505 raft_consensus.cc:482] T 52a667864bb2468e946bac9e732ed721 P 94f6ba3de346413088653c90bad52514 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 10:17:59.658713 11505 raft_consensus.cc:3037] T 52a667864bb2468e946bac9e732ed721 P 94f6ba3de346413088653c90bad52514 [term 0 FOLLOWER]: Advancing to term 1
I20260619 10:17:59.659524 11505 raft_consensus.cc:504] T 52a667864bb2468e946bac9e732ed721 P 94f6ba3de346413088653c90bad52514 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94f6ba3de346413088653c90bad52514" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 36095 } } peers { permanent_uuid: "f17a786d859d4913bdefb01bdd7323d3" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 38701 } } peers { permanent_uuid: "ba36208dba524eb494d861b98bc13670" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 39921 } }
I20260619 10:17:59.659652 11505 leader_election.cc:288] T 52a667864bb2468e946bac9e732ed721 P 94f6ba3de346413088653c90bad52514 [CANDIDATE]: Term 1 election: Requested vote from peers f17a786d859d4913bdefb01bdd7323d3 (127.0.206.67:38701), ba36208dba524eb494d861b98bc13670 (127.0.206.65:39921)
I20260619 10:17:59.659788 11439 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "52a667864bb2468e946bac9e732ed721" candidate_uuid: "94f6ba3de346413088653c90bad52514" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f17a786d859d4913bdefb01bdd7323d3"
I20260619 10:17:59.659821 11171 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "52a667864bb2468e946bac9e732ed721" candidate_uuid: "94f6ba3de346413088653c90bad52514" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba36208dba524eb494d861b98bc13670"
I20260619 10:17:59.659857 11439 raft_consensus.cc:3037] T 52a667864bb2468e946bac9e732ed721 P f17a786d859d4913bdefb01bdd7323d3 [term 0 FOLLOWER]: Advancing to term 1
I20260619 10:17:59.659922 11171 raft_consensus.cc:3037] T 52a667864bb2468e946bac9e732ed721 P ba36208dba524eb494d861b98bc13670 [term 0 FOLLOWER]: Advancing to term 1
I20260619 10:17:59.660549 11439 raft_consensus.cc:2454] T 52a667864bb2468e946bac9e732ed721 P f17a786d859d4913bdefb01bdd7323d3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94f6ba3de346413088653c90bad52514 in term 1.
I20260619 10:17:59.660698 11171 raft_consensus.cc:2454] T 52a667864bb2468e946bac9e732ed721 P ba36208dba524eb494d861b98bc13670 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94f6ba3de346413088653c90bad52514 in term 1.
I20260619 10:17:59.660845 11238 leader_election.cc:302] T 52a667864bb2468e946bac9e732ed721 P 94f6ba3de346413088653c90bad52514 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 94f6ba3de346413088653c90bad52514, ba36208dba524eb494d861b98bc13670; no voters:
I20260619 10:17:59.660946 11505 raft_consensus.cc:2781] T 52a667864bb2468e946bac9e732ed721 P 94f6ba3de346413088653c90bad52514 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 10:17:59.661095 11505 raft_consensus.cc:686] T 52a667864bb2468e946bac9e732ed721 P 94f6ba3de346413088653c90bad52514 [term 1 LEADER]: Becoming Leader. State: Replica: 94f6ba3de346413088653c90bad52514, State: Running, Role: LEADER
I20260619 10:17:59.661201 11505 consensus_queue.cc:237] T 52a667864bb2468e946bac9e732ed721 P 94f6ba3de346413088653c90bad52514 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94f6ba3de346413088653c90bad52514" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 36095 } } peers { permanent_uuid: "f17a786d859d4913bdefb01bdd7323d3" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 38701 } } peers { permanent_uuid: "ba36208dba524eb494d861b98bc13670" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 39921 } }
I20260619 10:17:59.662011 11025 catalog_manager.cc:5495] T 52a667864bb2468e946bac9e732ed721 P 94f6ba3de346413088653c90bad52514 reported cstate change: term changed from 0 to 1, leader changed from <none> to 94f6ba3de346413088653c90bad52514 (127.0.206.66). New cstate: current_term: 1 leader_uuid: "94f6ba3de346413088653c90bad52514" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94f6ba3de346413088653c90bad52514" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 36095 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f17a786d859d4913bdefb01bdd7323d3" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 38701 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ba36208dba524eb494d861b98bc13670" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 39921 } health_report { overall_health: UNKNOWN } } }
I20260619 10:17:59.674242 11495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:59.672333 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58682 (local address 127.0.206.66:36095)
0619 10:17:59.672379 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0619 10:17:59.672381 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 10:17:59.672389 (+ 8us) server_negotiation.cc:407] Connection header received
0619 10:17:59.672428 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:17:59.672429 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:17:59.672441 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:17:59.672497 (+ 56us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 10:17:59.672605 (+ 108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:59.673022 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:59.673666 (+ 644us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:17:59.673834 (+ 168us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:17:59.673880 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:17:59.673998 (+ 118us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 10:17:59.674079 (+ 81us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 10:17:59.674132 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0619 10:17:59.674165 (+ 33us) server_negotiation.cc:299] Negotiation successful
0619 10:17:59.674181 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
W20260619 10:17:59.675186 11352 tablet.cc:2369] T 52a667864bb2468e946bac9e732ed721 P 94f6ba3de346413088653c90bad52514: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 10:17:59.675832 11439 raft_consensus.cc:1261] T 52a667864bb2468e946bac9e732ed721 P f17a786d859d4913bdefb01bdd7323d3 [term 1 FOLLOWER]: Refusing update from remote peer 94f6ba3de346413088653c90bad52514: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 10:17:59.675832 11171 raft_consensus.cc:1261] T 52a667864bb2468e946bac9e732ed721 P ba36208dba524eb494d861b98bc13670 [term 1 FOLLOWER]: Refusing update from remote peer 94f6ba3de346413088653c90bad52514: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 10:17:59.676129 11505 consensus_queue.cc:1035] T 52a667864bb2468e946bac9e732ed721 P 94f6ba3de346413088653c90bad52514 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba36208dba524eb494d861b98bc13670" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 39921 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 10:17:59.676273 11509 consensus_queue.cc:1035] T 52a667864bb2468e946bac9e732ed721 P 94f6ba3de346413088653c90bad52514 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f17a786d859d4913bdefb01bdd7323d3" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 38701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 10:17:59.680270 11026 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51948:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 10:17:59.680358 11026 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51948:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 10:17:59.680977 11026 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 32ffb0e666e2497d9323388ca4461c47: Sending DeleteTablet for 3 replicas of tablet 52a667864bb2468e946bac9e732ed721
I20260619 10:17:59.681310 11419 tablet_service.cc:1514] Processing DeleteTablet for tablet 52a667864bb2468e946bac9e732ed721 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 10:17:59 UTC) from {username='kudu', principal='kudu/127.0.206.126@KRBTEST.COM'} at 127.0.0.1:37968
I20260619 10:17:59.681322 11151 tablet_service.cc:1514] Processing DeleteTablet for tablet 52a667864bb2468e946bac9e732ed721 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 10:17:59 UTC) from {username='kudu', principal='kudu/127.0.206.126@KRBTEST.COM'} at 127.0.0.1:42386
I20260619 10:17:59.681536 11523 tablet_replica.cc:330] T 52a667864bb2468e946bac9e732ed721 P f17a786d859d4913bdefb01bdd7323d3: stopping tablet replica
I20260619 10:17:59.681524 11285 tablet_service.cc:1514] Processing DeleteTablet for tablet 52a667864bb2468e946bac9e732ed721 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 10:17:59 UTC) from {username='kudu', principal='kudu/127.0.206.126@KRBTEST.COM'} at 127.0.0.1:58676
I20260619 10:17:59.681711 11523 raft_consensus.cc:2229] T 52a667864bb2468e946bac9e732ed721 P f17a786d859d4913bdefb01bdd7323d3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 10:17:59.681749 11524 tablet_replica.cc:330] T 52a667864bb2468e946bac9e732ed721 P 94f6ba3de346413088653c90bad52514: stopping tablet replica
I20260619 10:17:59.681852 11524 raft_consensus.cc:2229] T 52a667864bb2468e946bac9e732ed721 P 94f6ba3de346413088653c90bad52514 [term 1 LEADER]: Raft consensus shutting down.
I20260619 10:17:59.681926 825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTU9OP6/build/release/bin/kudu with pid 11088
I20260619 10:17:59.681995 11524 raft_consensus.cc:2258] T 52a667864bb2468e946bac9e732ed721 P 94f6ba3de346413088653c90bad52514 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 10:17:59.684504 11523 raft_consensus.cc:2258] T 52a667864bb2468e946bac9e732ed721 P f17a786d859d4913bdefb01bdd7323d3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 10:17:59.684862 11524 ts_tablet_manager.cc:1905] T 52a667864bb2468e946bac9e732ed721 P 94f6ba3de346413088653c90bad52514: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 10:17:59.684902 11523 ts_tablet_manager.cc:1905] T 52a667864bb2468e946bac9e732ed721 P f17a786d859d4913bdefb01bdd7323d3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 10:17:59.686460 11524 ts_tablet_manager.cc:1918] T 52a667864bb2468e946bac9e732ed721 P 94f6ba3de346413088653c90bad52514: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 10:17:59.686532 11524 log.cc:1196] T 52a667864bb2468e946bac9e732ed721 P 94f6ba3de346413088653c90bad52514: Deleting WAL directory at /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781864224577058-825-0/minicluster-data/ts-1/wal/wals/52a667864bb2468e946bac9e732ed721
I20260619 10:17:59.686843 11524 ts_tablet_manager.cc:1939] T 52a667864bb2468e946bac9e732ed721 P 94f6ba3de346413088653c90bad52514: Deleting consensus metadata
I20260619 10:17:59.687105 11523 ts_tablet_manager.cc:1918] T 52a667864bb2468e946bac9e732ed721 P f17a786d859d4913bdefb01bdd7323d3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 10:17:59.687242 11523 log.cc:1196] T 52a667864bb2468e946bac9e732ed721 P f17a786d859d4913bdefb01bdd7323d3: Deleting WAL directory at /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781864224577058-825-0/minicluster-data/ts-2/wal/wals/52a667864bb2468e946bac9e732ed721
I20260619 10:17:59.687228 11012 catalog_manager.cc:4841] TS 94f6ba3de346413088653c90bad52514 (127.0.206.66:36095): tablet 52a667864bb2468e946bac9e732ed721 (table test-table [id=fda22cca04d249e0a9705b7fec993a88]) successfully deleted
I20260619 10:17:59.687562 11523 ts_tablet_manager.cc:1939] T 52a667864bb2468e946bac9e732ed721 P f17a786d859d4913bdefb01bdd7323d3: Deleting consensus metadata
I20260619 10:17:59.687956 11012 catalog_manager.cc:4841] TS f17a786d859d4913bdefb01bdd7323d3 (127.0.206.67:38701): tablet 52a667864bb2468e946bac9e732ed721 (table test-table [id=fda22cca04d249e0a9705b7fec993a88]) successfully deleted
W20260619 10:17:59.688376 11010 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.0.206.65:39921 (error 108)
I20260619 10:17:59.688818 11496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:17:59.688673 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.65:39921 (local address 127.0.0.1:42396)
0619 10:17:59.688736 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0619 10:17:59.688785 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.65:39921: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260619 10:17:59.688961 825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTU9OP6/build/release/bin/kudu with pid 11222
W20260619 10:17:59.688948 11010 catalog_manager.cc:4584] TS ba36208dba524eb494d861b98bc13670 (127.0.206.65:39921): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 52a667864bb2468e946bac9e732ed721: Network error: Client connection negotiation failed: client connection to 127.0.206.65:39921: connect: Connection refused (error 111)
I20260619 10:17:59.693890 825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTU9OP6/build/release/bin/kudu with pid 11356
I20260619 10:17:59.698781 825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTU9OP6/build/release/bin/kudu with pid 10994
2026-06-19T10:17:59Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (1927 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781864224577058-825-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 10:17:59 dist-test-slave-6pt2 krb5kdc[11528](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 10:17:59 dist-test-slave-6pt2 krb5kdc[11528](info): set up 2 sockets
Jun 19 10:17:59 dist-test-slave-6pt2 krb5kdc[11528](info): commencing operation
krb5kdc: starting...
W20260619 10:18:00.707741 825 mini_kdc.cc:121] Time spent starting KDC: real 0.989s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781864224577058-825-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781864224577058-825-0/krb5kdc/test-user.keytab.
Jun 19 10:18:00 dist-test-slave-6pt2 krb5kdc[11528](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864280, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T10:18:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T10:18:00Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.0.206.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.0.206.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.0.206.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781864224577058-825-0/krb5kdc/rangeradmin_127.0.206.84@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.0.206.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781864224577058-825-0/krb5kdc/rangeradmin_127.0.206.84@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.0.206.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.0.206.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.0.206.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781864224577058-825-0/krb5kdc/rangerlookup_127.0.206.84@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.0.206.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781864224577058-825-0/krb5kdc/rangerlookup_127.0.206.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.0.206.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.0.206.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.0.206.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781864224577058-825-0/krb5kdc/HTTP_127.0.206.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.0.206.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781864224577058-825-0/krb5kdc/HTTP_127.0.206.84@KRBTEST.COM.keytab.
I20260619 10:18:00.858726 825 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781864224577058-825-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskTU9OP6/build/release/bin/postgres/pg_ctl -D /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781864224577058-825-0/minicluster-data/postgres -l logfile start
2026-06-19 10:18:03.120 UTC [11558] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-06-19 10:18:03.120 UTC [11558] LOG: listening on IPv4 address "127.0.206.84", port 33543
2026-06-19 10:18:03.124 UTC [11558] LOG: listening on Unix socket "/tmp/.s.PGSQL.33543"
2026-06-19 10:18:03.154 UTC [11561] LOG: database system was shut down at 2026-06-19 10:18:01 UTC
2026-06-19 10:18:03.158 UTC [11558] LOG: database system is ready to accept connections
I20260619 10:18:04.107530 825 mini_postgres.cc:96] Postgres bound to 33543
2026-06-19 10:18:04.113 UTC [11569] FATAL: database "slave" does not exist
127.0.206.84:33543 - accepting connections
I20260619 10:18:04.113940 825 mini_ranger.cc:161] Starting Ranger...
I20260619 10:18:04.128276 825 mini_ranger.cc:84] Created miniranger Postgres user
I20260619 10:18:04.534986 825 mini_ranger.cc:87] Created ranger Postgres database
I20260619 10:18:04.535048 825 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781864224577058-825-0/minicluster-data/ranger-admin
2026-06-19 10:18:04,801 [I] DB FLAVOR :POSTGRES
2026-06-19 10:18:04,801 [I] --------- Verifying Ranger DB connection ---------
2026-06-19 10:18:04,801 [I] Checking connection..
2026-06-19 10:18:04,801 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskTU9OP6/build/release/bin/postgresql.jar:/tmp/dist-test-taskTU9OP6/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.206.84:33543/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-06-19 10:18:05,058 [I] Checking connection passed.
2026-06-19 10:18:05,059 [I] --------- Verifying version history table ---------
2026-06-19 10:18:05,059 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskTU9OP6/build/release/bin/postgresql.jar:/tmp/dist-test-taskTU9OP6/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.206.84:33543/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-06-19 10:18:05,299 [I] Table x_db_version_h does not exist in database ranger
2026-06-19 10:18:05,299 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskTU9OP6/build/release/bin/postgresql.jar:/tmp/dist-test-taskTU9OP6/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.206.84:33543/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-06-19 10:18:05,521 [I] Table x_db_version_h does not exist in database ranger
2026-06-19 10:18:05,521 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-06-19 10:18:05,521 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskTU9OP6/build/release/bin/postgresql.jar:/tmp/dist-test-taskTU9OP6/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.206.84:33543/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskTU9OP6/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-06-19 10:18:05.740 UTC [11633] WARNING: there is no transaction in progress
2026-06-19 10:18:05,746 [I] create_dbversion_catalog.sql file imported successfully
2026-06-19 10:18:05,746 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskTU9OP6/build/release/bin/postgresql.jar:/tmp/dist-test-taskTU9OP6/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.206.84:33543/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-06-19 10:18:05,970 [I] Table x_db_version_h already exists in database 'ranger'
2026-06-19 10:18:05,970 [I] --------- Importing Ranger Core DB Schema ---------
2026-06-19 10:18:05,971 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskTU9OP6/build/release/bin/postgresql.jar:/tmp/dist-test-taskTU9OP6/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.206.84:33543/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-06-19 10:18:06,206 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskTU9OP6/build/release/bin/postgresql.jar:/tmp/dist-test-taskTU9OP6/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.206.84:33543/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-06-19 10:18:06,456 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskTU9OP6/build/release/bin/postgresql.jar:/tmp/dist-test-taskTU9OP6/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.206.84:33543/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-6pt2.c.gcp-upstream.internal','N') ;"
2026-06-19 10:18:06,698 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskTU9OP6/build/release/bin/postgresql.jar:/tmp/dist-test-taskTU9OP6/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.206.84:33543/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-06-19 10:18:06,919 [I] Table x_portal_user does not exist in database ranger
2026-06-19 10:18:06,920 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskTU9OP6/build/release/bin/postgresql.jar:/tmp/dist-test-taskTU9OP6/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.206.84:33543/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-06-19 10:18:07,137 [I] Table x_policy_ref_group does not exist in database ranger
2026-06-19 10:18:07,138 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskTU9OP6/build/release/bin/postgresql.jar:/tmp/dist-test-taskTU9OP6/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.206.84:33543/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-06-19 10:18:07,388 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskTU9OP6/build/release/bin/postgresql.jar:/tmp/dist-test-taskTU9OP6/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.206.84:33543/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-06-19 10:18:07,636 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-06-19 10:18:07,636 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskTU9OP6/build/release/bin/postgresql.jar:/tmp/dist-test-taskTU9OP6/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.206.84:33543/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskTU9OP6/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-06-19 10:18:08.525 UTC [11741] WARNING: there is no transaction in progress
2026-06-19 10:18:08.540 UTC [11741] WARNING: there is no transaction in progress
2026-06-19 10:18:08.555 UTC [11741] WARNING: there is no transaction in progress
2026-06-19 10:18:08.568 UTC [11741] WARNING: there is no transaction in progress
2026-06-19 10:18:08.582 UTC [11741] WARNING: there is no transaction in progress
2026-06-19 10:18:08.598 UTC [11741] WARNING: there is no transaction in progress
2026-06-19 10:18:08.659 UTC [11741] WARNING: there is no transaction in progress
2026-06-19 10:18:08.665 UTC [11741] WARNING: there is no transaction in progress
2026-06-19 10:18:08.678 UTC [11741] WARNING: there is no transaction in progress
2026-06-19 10:18:08.689 UTC [11741] WARNING: there is no transaction in progress
2026-06-19 10:18:08.699 UTC [11741] WARNING: there is no transaction in progress
2026-06-19 10:18:08.713 UTC [11741] WARNING: there is no transaction in progress
2026-06-19 10:18:08.722 UTC [11741] WARNING: there is no transaction in progress
2026-06-19 10:18:08.728 UTC [11741] WARNING: there is no transaction in progress
2026-06-19 10:18:08.733 UTC [11741] WARNING: there is no transaction in progress
2026-06-19 10:18:08.739 UTC [11741] WARNING: there is no transaction in progress
2026-06-19 10:18:09,297 [I] ranger_core_db_postgres.sql file imported successfully
2026-06-19 10:18:09,298 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskTU9OP6/build/release/bin/postgresql.jar:/tmp/dist-test-taskTU9OP6/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.206.84:33543/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-06-19 10:18:09,512 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-06-19 10:18:09,512 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskTU9OP6/build/release/bin/postgresql.jar:/tmp/dist-test-taskTU9OP6/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.206.84:33543/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-06-19 10:18:09,732 [I] Table x_portal_user already exists in database 'ranger'
2026-06-19 10:18:09,733 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskTU9OP6/build/release/bin/postgresql.jar:/tmp/dist-test-taskTU9OP6/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.206.84:33543/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-06-19 10:18:09,962 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-06-19 10:18:09,962 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskTU9OP6/build/release/bin/postgresql.jar:/tmp/dist-test-taskTU9OP6/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.206.84:33543/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-06-19 10:18:10,183 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskTU9OP6/build/release/bin/postgresql.jar:/tmp/dist-test-taskTU9OP6/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.206.84:33543/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-06-19 10:18:10,435 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskTU9OP6/build/release/bin/postgresql.jar:/tmp/dist-test-taskTU9OP6/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.206.84:33543/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-6pt2.c.gcp-upstream.internal';"
2026-06-19 10:18:10,670 [I] CORE_DB_SCHEMA import status has been updated
2026-06-19 10:18:10,671 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskTU9OP6/build/release/bin/postgresql.jar:/tmp/dist-test-taskTU9OP6/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.206.84:33543/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-06-19 10:18:10,919 [I] DB_PATCHES have already been applied
I20260619 10:18:10.925436 825 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781864224577058-825-0/minicluster-data/ranger-admin:/tmp/dist-test-taskTU9OP6/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskTU9OP6/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskTU9OP6/build/release/bin/postgresql.jar:/tmp/dist-test-taskTU9OP6/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260619 10:18:10.925539 825 mini_ranger.cc:193] Using host: 127.0.206.84
I20260619 10:18:10.928437 825 mini_ranger.cc:227] Ranger admin URL: http://127.0.206.84:35745
Jun 19, 2026 10:18:11 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Jun 19, 2026 10:18:11 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Jun 19, 2026 10:18:11 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781864224577058-825-0/minicluster-data/ranger-admin/ews/webapp
Jun 19, 2026 10:18:11 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781864224577058-825-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Jun 19, 2026 10:18:11 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781864224577058-825-0/minicluster-data/ranger-admin/ews/webapp] .....
Jun 19, 2026 10:18:11 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781864224577058-825-0/minicluster-data/ranger-admin/ews/webapp].
Jun 19, 2026 10:18:11 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260619 10:18:12.046702 825 mini_ranger.cc:160] Time spent starting Ranger: real 7.933s user 0.001s sys 0.007s
Jun 19, 2026 10:18:31 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Jun 19, 2026 10:18:31 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
Jun 19, 2026 10:18:31 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Jun 19, 2026 10:18:31 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Jun 19, 2026 10:18:31 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Jun 19, 2026 10:18:31 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Jun 19, 2026 10:18:31 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Jun 19, 2026 10:18:31 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Jun 19, 2026 10:18:31 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Jun 19, 2026 10:18:31 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Jun 19, 2026 10:18:31 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Jun 19, 2026 10:18:31 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Jun 19, 2026 10:18:31 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Jun 19, 2026 10:18:31 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Jun 19, 2026 10:18:31 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Jun 19, 2026 10:18:31 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Jun 19, 2026 10:18:31 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Jun 19, 2026 10:18:32 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260619 10:18:33.357116 825 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.0.206.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.0.206.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.0.206.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781864224577058-825-0/krb5kdc/rangerkms_127.0.206.84@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.0.206.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781864224577058-825-0/krb5kdc/rangerkms_127.0.206.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.0.206.84@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.0.206.84@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.0.206.84@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781864224577058-825-0/krb5kdc/HTTP_127.0.206.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.0.206.84@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781864224577058-825-0/krb5kdc/HTTP_127.0.206.84@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Jun 19 10:18:33 dist-test-slave-6pt2 krb5kdc[11528](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864313, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260619 10:18:33.431535 825 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260619 10:18:33.440374 825 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260619 10:18:34.214178 825 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260619 10:18:34.214243 825 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781864224577058-825-0/minicluster-data/ranger-kms
I20260619 10:18:34.214255 825 mini_ranger_kms.cc:227] Using postgres at 127.0.206.84:33543
2026-06-19 10:18:34,389 [I] DB FLAVOR :POSTGRES
2026-06-19 10:18:34,390 [I] --------- Verifying Ranger DB connection ---------
2026-06-19 10:18:34,390 [I] Checking connection
2026-06-19 10:18:34,390 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskTU9OP6/build/release/bin/postgresql.jar:/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781864224577058-825-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.206.84:33543/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-06-19 10:18:34,625 [I] connection success
2026-06-19 10:18:34,626 [I] --------- Verifying Ranger DB tables ---------
2026-06-19 10:18:34,626 [I] Verifying table ranger_masterkey in database rangerkms
2026-06-19 10:18:34,626 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskTU9OP6/build/release/bin/postgresql.jar:/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781864224577058-825-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.206.84:33543/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-06-19 10:18:34,856 [I] Table ranger_masterkey does not exist in database rangerkms
2026-06-19 10:18:34,856 [I] --------- Importing Ranger Core DB Schema ---------
2026-06-19 10:18:34,856 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-06-19 10:18:34,857 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskTU9OP6/build/release/bin/postgresql.jar:/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781864224577058-825-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.206.84:33543/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781864224577058-825-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-06-19 10:18:35,113 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260619 10:18:35.563280 825 mini_ranger_kms.cc:310] Created kms service
I20260619 10:18:35.726382 825 mini_ranger_kms.cc:326] Created kudu user
I20260619 10:18:35.766103 825 mini_ranger_kms.cc:343] Created rangerkms user
Jun 19 10:18:35 dist-test-slave-6pt2 krb5kdc[11528](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864313, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.0.206.84@KRBTEST.COM
I20260619 10:18:36.156504 825 mini_ranger_kms.cc:384] Added ranger policy
I20260619 10:18:36.156620 825 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781864224577058-825-0/minicluster-data/ranger-kms:/tmp/dist-test-taskTU9OP6/build/release/bin/postgresql.jar:/tmp/dist-test-taskTU9OP6/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskTU9OP6/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260619 10:18:36.156647 825 mini_ranger_kms.cc:240] Using host: 127.0.206.84
I20260619 10:18:36.160409 825 mini_ranger_kms.cc:276] Ranger KMS PID: 11921
I20260619 10:18:36.160516 825 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.0.206.84:52435
Jun 19, 2026 10:18:37 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Jun 19, 2026 10:18:37 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Jun 19, 2026 10:18:37 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781864224577058-825-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Jun 19, 2026 10:18:37 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781864224577058-825-0/minicluster-data/ranger-kms/ews/webapp] .....
Jun 19, 2026 10:18:37 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781864224577058-825-0/minicluster-data/ranger-kms/ews/webapp].
Jun 19, 2026 10:18:37 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
I20260619 10:18:38.399756 825 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 4.968s user 0.000s sys 0.158s
I20260619 10:18:38.399864 825 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260619 10:18:38.399933 825 mini_ranger_kms.cc:401] 127.0.206.84:52435/kms/v1/keys
Jun 19 10:18:38 dist-test-slave-6pt2 krb5kdc[11528](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1781864318, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.0.206.84@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Jun 19 10:18:38 dist-test-slave-6pt2 krb5kdc[11528](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864318, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.0.206.84@KRBTEST.COM for HTTP/127.0.206.84@KRBTEST.COM
Jun 19 10:18:39 dist-test-slave-6pt2 krb5kdc[11528](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864318, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.0.206.84@KRBTEST.COM for HTTP/127.0.206.84@KRBTEST.COM
Jun 19 10:18:41 dist-test-slave-6pt2 krb5kdc[11528](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864321, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.0.206.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.206.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.206.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781864224577058-825-0/krb5kdc/kudu_127.0.206.126.keytab.
Entry for principal kudu/127.0.206.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781864224577058-825-0/krb5kdc/kudu_127.0.206.126.keytab.
I20260619 10:18:41.399269 825 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTU9OP6/build/release/bin/kudu
/tmp/dist-test-taskTU9OP6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781864224577058-825-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781864224577058-825-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781864224577058-825-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781864224577058-825-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.206.126:35047
--webserver_interface=127.0.206.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781864224577058-825-0/krb5kdc/kudu_127.0.206.126.keytab
--principal=kudu/127.0.206.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.206.84:43823
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.206.126:35047
--ranger_config_path=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781864224577058-825-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.0.206.84:52435/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781864224577058-825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781864224577058-825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781864224577058-825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 10:18:41.498468 11972 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 10:18:41.498726 11972 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 10:18:41.498751 11972 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 10:18:41.499881 11972 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 10:18:41.499920 11972 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 10:18:41.499934 11972 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 10:18:41.499948 11972 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 10:18:41.499960 11972 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 10:18:41.501461 11972 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.206.84:43823
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781864224577058-825-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781864224577058-825-0/minicluster-data/master-0/wal
--ranger_kms_url=127.0.206.84:52435/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.0.206.126:35047
--ranger_config_path=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781864224577058-825-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781864224577058-825-0/krb5kdc/kudu_127.0.206.126.keytab
--principal=kudu/127.0.206.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.206.126:35047
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781864224577058-825-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.206.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11972
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781864224577058-825-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 09:32:44 UTC on ccycloud.quasar-gbtldp.root.comops.site
build id 4455440
I20260619 10:18:41.501691 11972 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 10:18:41.501952 11972 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 10:18:41.507339 11978 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 10:18:41.507477 11972 server_base.cc:1034] running on GCE node
W20260619 10:18:41.507316 11977 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 10:18:41.507611 11980 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 10:18:41.507871 11972 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 10:18:41.508216 11972 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 10:18:41.509385 11972 hybrid_clock.cc:648] HybridClock initialized: now 1781864321509363 us; error 33 us; skew 500 ppm
Jun 19 10:18:41 dist-test-slave-6pt2 krb5kdc[11528](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864321, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 10:18:41.512373 11972 init.cc:377] Logged in from keytab as kudu/127.0.206.126@KRBTEST.COM (short username kudu)
I20260619 10:18:41.512854 11972 webserver.cc:460] Webserver started at http://127.0.206.126:42029/ using document root <none> and password file <none>
I20260619 10:18:41.513089 11972 fs_manager.cc:359] Metadata directory not provided
I20260619 10:18:41.513155 11972 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 10:18:41.513314 11972 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Jun 19 10:18:41 dist-test-slave-6pt2 krb5kdc[11528](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864321, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.126@KRBTEST.COM for HTTP/127.0.206.84@KRBTEST.COM
I20260619 10:18:41.567209 11972 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781864224577058-825-0/minicluster-data/master-0/data/instance:
uuid: "a2115a1d164b42e49e7d16df95ca9294"
format_stamp: "Formatted at 2026-06-19 10:18:41 on dist-test-slave-6pt2"
server_key: "9f4f911eab25c43336ad20a28ef8715e"
server_key_iv: "da6af42f730e9e09e51658de26c399cb"
server_key_version: "kuduclusterkey@0"
I20260619 10:18:41.567663 11972 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781864224577058-825-0/minicluster-data/master-0/wal/instance:
uuid: "a2115a1d164b42e49e7d16df95ca9294"
format_stamp: "Formatted at 2026-06-19 10:18:41 on dist-test-slave-6pt2"
server_key: "9f4f911eab25c43336ad20a28ef8715e"
server_key_iv: "da6af42f730e9e09e51658de26c399cb"
server_key_version: "kuduclusterkey@0"
I20260619 10:18:41.569401 11972 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.001s
I20260619 10:18:41.581683 11987 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 10:18:41.582103 11972 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 10:18:41.582212 11972 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781864224577058-825-0/minicluster-data/master-0/data,/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781864224577058-825-0/minicluster-data/master-0/wal
uuid: "a2115a1d164b42e49e7d16df95ca9294"
format_stamp: "Formatted at 2026-06-19 10:18:41 on dist-test-slave-6pt2"
server_key: "9f4f911eab25c43336ad20a28ef8715e"
server_key_iv: "da6af42f730e9e09e51658de26c399cb"
server_key_version: "kuduclusterkey@0"
I20260619 10:18:41.582340 11972 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781864224577058-825-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781864224577058-825-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781864224577058-825-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 10:18:41.606534 11972 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 10:18:41.608762 11972 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 10:18:41.608934 11972 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 10:18:41.613983 11972 rpc_server.cc:307] RPC server started. Bound to: 127.0.206.126:35047
I20260619 10:18:41.614471 12039 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.206.126:35047 every 8 connection(s)
I20260619 10:18:41.614840 11972 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781864224577058-825-0/minicluster-data/master-0/data/info.pb
I20260619 10:18:41.615499 12040 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 10:18:41.615737 825 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTU9OP6/build/release/bin/kudu as pid 11972
I20260619 10:18:41.615854 825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781864224577058-825-0/minicluster-data/master-0/wal/instance
I20260619 10:18:41.616070 825 external_mini_cluster.cc:1428] Setting key b565bb34810fee191c870a88a4d25b74
I20260619 10:18:41.618994 12040 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a2115a1d164b42e49e7d16df95ca9294: Bootstrap starting.
I20260619 10:18:41.619695 12040 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a2115a1d164b42e49e7d16df95ca9294: Neither blocks nor log segments found. Creating new log.
I20260619 10:18:41.620054 12040 log.cc:824] T 00000000000000000000000000000000 P a2115a1d164b42e49e7d16df95ca9294: Log is configured to *not* fsync() on all Append() calls
I20260619 10:18:41.620746 12040 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a2115a1d164b42e49e7d16df95ca9294: No bootstrap required, opened a new log
I20260619 10:18:41.623154 12040 raft_consensus.cc:348] T 00000000000000000000000000000000 P a2115a1d164b42e49e7d16df95ca9294 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a2115a1d164b42e49e7d16df95ca9294" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 35047 } }
I20260619 10:18:41.623413 12040 raft_consensus.cc:374] T 00000000000000000000000000000000 P a2115a1d164b42e49e7d16df95ca9294 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 10:18:41.623533 12040 raft_consensus.cc:729] T 00000000000000000000000000000000 P a2115a1d164b42e49e7d16df95ca9294 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2115a1d164b42e49e7d16df95ca9294, State: Initialized, Role: FOLLOWER
I20260619 10:18:41.623746 12040 consensus_queue.cc:260] T 00000000000000000000000000000000 P a2115a1d164b42e49e7d16df95ca9294 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a2115a1d164b42e49e7d16df95ca9294" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 35047 } }
Jun 19 10:18:41 dist-test-slave-6pt2 krb5kdc[11528](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864321, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.206.126@KRBTEST.COM
I20260619 10:18:41.623909 12040 raft_consensus.cc:388] T 00000000000000000000000000000000 P a2115a1d164b42e49e7d16df95ca9294 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 10:18:41.623952 12040 raft_consensus.cc:482] T 00000000000000000000000000000000 P a2115a1d164b42e49e7d16df95ca9294 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 10:18:41.624002 12040 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a2115a1d164b42e49e7d16df95ca9294 [term 0 FOLLOWER]: Advancing to term 1
I20260619 10:18:41.624572 12040 raft_consensus.cc:504] T 00000000000000000000000000000000 P a2115a1d164b42e49e7d16df95ca9294 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a2115a1d164b42e49e7d16df95ca9294" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 35047 } }
I20260619 10:18:41.624680 12040 leader_election.cc:302] T 00000000000000000000000000000000 P a2115a1d164b42e49e7d16df95ca9294 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a2115a1d164b42e49e7d16df95ca9294; no voters:
I20260619 10:18:41.624841 12040 leader_election.cc:288] T 00000000000000000000000000000000 P a2115a1d164b42e49e7d16df95ca9294 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 10:18:41.625054 12046 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a2115a1d164b42e49e7d16df95ca9294 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 10:18:41.625533 12046 raft_consensus.cc:686] T 00000000000000000000000000000000 P a2115a1d164b42e49e7d16df95ca9294 [term 1 LEADER]: Becoming Leader. State: Replica: a2115a1d164b42e49e7d16df95ca9294, State: Running, Role: LEADER
I20260619 10:18:41.625144 12040 sys_catalog.cc:564] T 00000000000000000000000000000000 P a2115a1d164b42e49e7d16df95ca9294 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 10:18:41.625648 12046 consensus_queue.cc:237] T 00000000000000000000000000000000 P a2115a1d164b42e49e7d16df95ca9294 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a2115a1d164b42e49e7d16df95ca9294" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 35047 } }
I20260619 10:18:41.627558 12047 sys_catalog.cc:455] T 00000000000000000000000000000000 P a2115a1d164b42e49e7d16df95ca9294 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a2115a1d164b42e49e7d16df95ca9294. Latest consensus state: current_term: 1 leader_uuid: "a2115a1d164b42e49e7d16df95ca9294" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a2115a1d164b42e49e7d16df95ca9294" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 35047 } } }
I20260619 10:18:41.627660 12047 sys_catalog.cc:458] T 00000000000000000000000000000000 P a2115a1d164b42e49e7d16df95ca9294 [sys.catalog]: This master's current role is: LEADER
I20260619 10:18:41.628051 12045 sys_catalog.cc:455] T 00000000000000000000000000000000 P a2115a1d164b42e49e7d16df95ca9294 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a2115a1d164b42e49e7d16df95ca9294" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a2115a1d164b42e49e7d16df95ca9294" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 35047 } } }
I20260619 10:18:41.628149 12045 sys_catalog.cc:458] T 00000000000000000000000000000000 P a2115a1d164b42e49e7d16df95ca9294 [sys.catalog]: This master's current role is: LEADER
I20260619 10:18:41.628229 12043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:18:41.616625 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56668 (local address 127.0.206.126:35047)
0619 10:18:41.616850 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0619 10:18:41.616861 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0619 10:18:41.616893 (+ 32us) server_negotiation.cc:407] Connection header received
0619 10:18:41.617364 (+ 471us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:18:41.617374 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:18:41.617447 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:18:41.617606 (+ 159us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:18:41.618388 (+ 782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:18:41.619768 (+ 1380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:18:41.620822 (+ 1054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:18:41.621454 (+ 632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:18:41.621566 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:18:41.624526 (+ 2960us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 10:18:41.624531 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 10:18:41.624531 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 10:18:41.624543 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0619 10:18:41.626498 (+ 1955us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:18:41.626919 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:18:41.626922 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:18:41.626922 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:18:41.627023 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:18:41.627235 (+ 212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:18:41.627236 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:18:41.627236 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:18:41.627577 (+ 341us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 10:18:41.627763 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0619 10:18:41.627840 (+ 77us) server_negotiation.cc:299] Negotiation successful
0619 10:18:41.627977 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":176,"thread_start_us":92,"threads_started":1}
I20260619 10:18:41.628216 12040 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781864224577058-825-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260619 10:18:41.634749 12059 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 10:18:41.635401 12059 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 10:18:41.639137 12059 catalog_manager.cc:1347] Generated new cluster ID: bca591ae9e7244e798a73a99bfa38eb9
I20260619 10:18:41.639273 12059 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 10:18:41.650945 12059 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 10:18:41.651742 12059 catalog_manager.cc:1504] Loading token signing keys...
I20260619 10:18:41.658201 12059 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a2115a1d164b42e49e7d16df95ca9294: Generated new TSK 0
I20260619 10:18:41.658500 12059 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.0.206.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.206.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.206.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781864224577058-825-0/krb5kdc/kudu_127.0.206.65.keytab.
Entry for principal kudu/127.0.206.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781864224577058-825-0/krb5kdc/kudu_127.0.206.65.keytab.
I20260619 10:18:42.430698 825 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTU9OP6/build/release/bin/kudu
/tmp/dist-test-taskTU9OP6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781864224577058-825-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781864224577058-825-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781864224577058-825-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781864224577058-825-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.0.206.84:52435/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.206.65:0
--local_ip_for_outbound_sockets=127.0.206.65
--webserver_interface=127.0.206.65
--webserver_port=0
--tserver_master_addrs=127.0.206.126:35047
--keytab_file=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781864224577058-825-0/krb5kdc/kudu_127.0.206.65.keytab
--principal=kudu/127.0.206.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.206.84:43823
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781864224577058-825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781864224577058-825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781864224577058-825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 10:18:42.549183 12085 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 10:18:42.549540 12085 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 10:18:42.549607 12085 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 10:18:42.551255 12085 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 10:18:42.551312 12085 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 10:18:42.551352 12085 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.206.65
I20260619 10:18:42.553458 12085 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.206.84:43823
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781864224577058-825-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781864224577058-825-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.0.206.84:52435/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781864224577058-825-0/krb5kdc/kudu_127.0.206.65.keytab
--principal=kudu/127.0.206.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.206.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781864224577058-825-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.206.65
--webserver_port=0
--tserver_master_addrs=127.0.206.126:35047
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12085
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.206.65
--log_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781864224577058-825-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 09:32:44 UTC on ccycloud.quasar-gbtldp.root.comops.site
build id 4455440
I20260619 10:18:42.553722 12085 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 10:18:42.554064 12085 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 10:18:42.560441 12093 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 10:18:42.560513 12091 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 10:18:42.560720 12090 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 10:18:42.562139 12085 server_base.cc:1034] running on GCE node
I20260619 10:18:42.562350 12085 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 10:18:42.562759 12085 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 10:18:42.563916 12085 hybrid_clock.cc:648] HybridClock initialized: now 1781864322563883 us; error 46 us; skew 500 ppm
Jun 19 10:18:42 dist-test-slave-6pt2 krb5kdc[11528](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864322, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 10:18:42.566906 12085 init.cc:377] Logged in from keytab as kudu/127.0.206.65@KRBTEST.COM (short username kudu)
I20260619 10:18:42.567358 12085 webserver.cc:460] Webserver started at http://127.0.206.65:39779/ using document root <none> and password file <none>
I20260619 10:18:42.567538 12085 fs_manager.cc:359] Metadata directory not provided
I20260619 10:18:42.567571 12085 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 10:18:42.567659 12085 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Jun 19 10:18:42 dist-test-slave-6pt2 krb5kdc[11528](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864322, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.65@KRBTEST.COM for HTTP/127.0.206.84@KRBTEST.COM
I20260619 10:18:42.581920 12085 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781864224577058-825-0/minicluster-data/ts-0/data/instance:
uuid: "b461caa3dc3741278f936964a3edfe88"
format_stamp: "Formatted at 2026-06-19 10:18:42 on dist-test-slave-6pt2"
server_key: "f4b7feb2d621b14f2aaf8d95219a154c"
server_key_iv: "87c0ca63d5d7e83844c520bbbe28bb8d"
server_key_version: "kuduclusterkey@0"
I20260619 10:18:42.582487 12085 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781864224577058-825-0/minicluster-data/ts-0/wal/instance:
uuid: "b461caa3dc3741278f936964a3edfe88"
format_stamp: "Formatted at 2026-06-19 10:18:42 on dist-test-slave-6pt2"
server_key: "f4b7feb2d621b14f2aaf8d95219a154c"
server_key_iv: "87c0ca63d5d7e83844c520bbbe28bb8d"
server_key_version: "kuduclusterkey@0"
I20260619 10:18:42.587581 12085 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.000s
I20260619 10:18:42.602484 12102 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 10:18:42.602828 12085 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 10:18:42.602941 12085 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781864224577058-825-0/minicluster-data/ts-0/data,/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781864224577058-825-0/minicluster-data/ts-0/wal
uuid: "b461caa3dc3741278f936964a3edfe88"
format_stamp: "Formatted at 2026-06-19 10:18:42 on dist-test-slave-6pt2"
server_key: "f4b7feb2d621b14f2aaf8d95219a154c"
server_key_iv: "87c0ca63d5d7e83844c520bbbe28bb8d"
server_key_version: "kuduclusterkey@0"
I20260619 10:18:42.603030 12085 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781864224577058-825-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781864224577058-825-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781864224577058-825-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 10:18:42.634279 12085 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 10:18:42.636915 12085 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 10:18:42.637118 12085 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 10:18:42.637372 12085 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 10:18:42.637774 12085 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 10:18:42.637871 12085 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 10:18:42.637970 12085 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 10:18:42.638042 12085 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 10:18:42.644784 12085 rpc_server.cc:307] RPC server started. Bound to: 127.0.206.65:34117
I20260619 10:18:42.645259 12085 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781864224577058-825-0/minicluster-data/ts-0/data/info.pb
I20260619 10:18:42.656440 825 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTU9OP6/build/release/bin/kudu as pid 12085
I20260619 10:18:42.656566 825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781864224577058-825-0/minicluster-data/ts-0/wal/instance
I20260619 10:18:42.656813 825 external_mini_cluster.cc:1428] Setting key de9dd498fc0b9b650085a7bf0bb03f66
I20260619 10:18:42.659010 12215 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.206.65:34117 every 8 connection(s)
Jun 19 10:18:42 dist-test-slave-6pt2 krb5kdc[11528](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864322, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.65@KRBTEST.COM for kudu/127.0.206.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.0.206.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.206.66@KRBTEST.COM" created.
I20260619 10:18:42.682801 12219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:18:42.659790 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:35047 (local address 127.0.206.65:53293)
0619 10:18:42.669312 (+ 9522us) negotiation.cc:107] Waiting for socket to connect
0619 10:18:42.669380 (+ 68us) client_negotiation.cc:174] Beginning negotiation
0619 10:18:42.670075 (+ 695us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 10:18:42.673162 (+ 3087us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 10:18:42.673168 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 10:18:42.673441 (+ 273us) client_negotiation.cc:189] Negotiated authn=SASL
0619 10:18:42.673743 (+ 302us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:18:42.673748 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:18:42.674876 (+ 1128us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:18:42.674877 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:18:42.676175 (+ 1298us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:18:42.676178 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:18:42.676763 (+ 585us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:18:42.676764 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:18:42.676931 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:18:42.677572 (+ 641us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 10:18:42.677582 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0619 10:18:42.679086 (+ 1504us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 10:18:42.681168 (+ 2082us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 10:18:42.681175 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 10:18:42.681178 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 10:18:42.681411 (+ 233us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 10:18:42.681641 (+ 230us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 10:18:42.681642 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 10:18:42.681642 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 10:18:42.681772 (+ 130us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 10:18:42.682046 (+ 274us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 10:18:42.682052 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 10:18:42.682227 (+ 175us) client_negotiation.cc:769] Sending connection context
0619 10:18:42.682353 (+ 126us) client_negotiation.cc:240] Negotiation successful
0619 10:18:42.682530 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9461,"thread_start_us":141,"threads_started":1}
I20260619 10:18:42.683303 12220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:18:42.660142 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.65:53293 (local address 127.0.206.126:35047)
0619 10:18:42.669206 (+ 9064us) server_negotiation.cc:206] Beginning negotiation
0619 10:18:42.669209 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 10:18:42.672872 (+ 3663us) server_negotiation.cc:407] Connection header received
0619 10:18:42.672910 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:18:42.672911 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:18:42.672928 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:18:42.672999 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:18:42.673879 (+ 880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:18:42.674752 (+ 873us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:18:42.676277 (+ 1525us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:18:42.676608 (+ 331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:18:42.676670 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:18:42.679185 (+ 2515us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 10:18:42.679187 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 10:18:42.679187 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 10:18:42.679195 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 10:18:42.681087 (+ 1892us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:18:42.681503 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:18:42.681504 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:18:42.681505 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:18:42.681564 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:18:42.681854 (+ 290us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:18:42.681855 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:18:42.681855 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:18:42.681978 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 10:18:42.682055 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0619 10:18:42.683156 (+ 1101us) server_negotiation.cc:299] Negotiation successful
0619 10:18:42.683234 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9035,"thread_start_us":50,"threads_started":1}
I20260619 10:18:42.683676 12216 heartbeater.cc:347] Connected to a master server at 127.0.206.126:35047
I20260619 10:18:42.683804 12216 heartbeater.cc:464] Registering TS with master...
I20260619 10:18:42.684016 12216 heartbeater.cc:511] Master 127.0.206.126:35047 requested a full tablet report, sending...
I20260619 10:18:42.684561 12004 ts_manager.cc:194] Registered new tserver with Master: b461caa3dc3741278f936964a3edfe88 (127.0.206.65:34117)
I20260619 10:18:42.685495 12004 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.206.65@KRBTEST.COM'} at 127.0.206.65:53293
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.206.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781864224577058-825-0/krb5kdc/kudu_127.0.206.66.keytab.
Entry for principal kudu/127.0.206.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781864224577058-825-0/krb5kdc/kudu_127.0.206.66.keytab.
I20260619 10:18:42.711565 825 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTU9OP6/build/release/bin/kudu
/tmp/dist-test-taskTU9OP6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781864224577058-825-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781864224577058-825-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781864224577058-825-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781864224577058-825-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.0.206.84:52435/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.206.66:0
--local_ip_for_outbound_sockets=127.0.206.66
--webserver_interface=127.0.206.66
--webserver_port=0
--tserver_master_addrs=127.0.206.126:35047
--keytab_file=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781864224577058-825-0/krb5kdc/kudu_127.0.206.66.keytab
--principal=kudu/127.0.206.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.206.84:43823
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781864224577058-825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781864224577058-825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781864224577058-825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Jun 19 10:18:42 dist-test-slave-6pt2 krb5kdc[11528](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1781864322, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260619 10:18:42.838054 12224 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 10:18:42.838310 12224 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 10:18:42.838343 12224 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 10:18:42.840015 12224 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 10:18:42.840070 12224 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 10:18:42.840106 12224 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.206.66
I20260619 10:18:42.842168 12224 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.206.84:43823
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781864224577058-825-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781864224577058-825-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.0.206.84:52435/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781864224577058-825-0/krb5kdc/kudu_127.0.206.66.keytab
--principal=kudu/127.0.206.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.206.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781864224577058-825-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.206.66
--webserver_port=0
--tserver_master_addrs=127.0.206.126:35047
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12224
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.206.66
--log_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781864224577058-825-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 09:32:44 UTC on ccycloud.quasar-gbtldp.root.comops.site
build id 4455440
I20260619 10:18:42.842519 12224 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 10:18:42.842809 12224 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 10:18:42.849265 12231 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 10:18:42.849543 12230 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 10:18:42.849659 12233 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 10:18:42.850589 12224 server_base.cc:1034] running on GCE node
I20260619 10:18:42.850845 12224 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 10:18:42.851116 12224 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 10:18:42.852299 12224 hybrid_clock.cc:648] HybridClock initialized: now 1781864322852278 us; error 37 us; skew 500 ppm
Jun 19 10:18:42 dist-test-slave-6pt2 krb5kdc[11528](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864322, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 10:18:42.855517 12224 init.cc:377] Logged in from keytab as kudu/127.0.206.66@KRBTEST.COM (short username kudu)
I20260619 10:18:42.856035 12224 webserver.cc:460] Webserver started at http://127.0.206.66:42423/ using document root <none> and password file <none>
I20260619 10:18:42.856343 12224 fs_manager.cc:359] Metadata directory not provided
I20260619 10:18:42.856406 12224 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 10:18:42.856562 12224 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Jun 19 10:18:42 dist-test-slave-6pt2 krb5kdc[11528](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864322, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.66@KRBTEST.COM for HTTP/127.0.206.84@KRBTEST.COM
I20260619 10:18:42.871287 12224 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781864224577058-825-0/minicluster-data/ts-1/data/instance:
uuid: "d7c17b58dde54337ba12510a19f60697"
format_stamp: "Formatted at 2026-06-19 10:18:42 on dist-test-slave-6pt2"
server_key: "eab94b970807bb22d2bf3510e26de7cc"
server_key_iv: "0d4113eb712f94b1db2668f9d1141b09"
server_key_version: "kuduclusterkey@0"
I20260619 10:18:42.871752 12224 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781864224577058-825-0/minicluster-data/ts-1/wal/instance:
uuid: "d7c17b58dde54337ba12510a19f60697"
format_stamp: "Formatted at 2026-06-19 10:18:42 on dist-test-slave-6pt2"
server_key: "eab94b970807bb22d2bf3510e26de7cc"
server_key_iv: "0d4113eb712f94b1db2668f9d1141b09"
server_key_version: "kuduclusterkey@0"
I20260619 10:18:42.873615 12224 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.000s
I20260619 10:18:42.898756 12240 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 10:18:42.898970 12224 fs_manager.cc:725] Time spent opening block manager: real 0.014s user 0.001s sys 0.000s
I20260619 10:18:42.899056 12224 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781864224577058-825-0/minicluster-data/ts-1/data,/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781864224577058-825-0/minicluster-data/ts-1/wal
uuid: "d7c17b58dde54337ba12510a19f60697"
format_stamp: "Formatted at 2026-06-19 10:18:42 on dist-test-slave-6pt2"
server_key: "eab94b970807bb22d2bf3510e26de7cc"
server_key_iv: "0d4113eb712f94b1db2668f9d1141b09"
server_key_version: "kuduclusterkey@0"
I20260619 10:18:42.899163 12224 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781864224577058-825-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781864224577058-825-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781864224577058-825-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 10:18:42.926414 12224 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 10:18:42.928964 12224 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 10:18:42.929200 12224 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 10:18:42.929504 12224 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 10:18:42.929950 12224 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 10:18:42.930052 12224 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 10:18:42.930130 12224 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 10:18:42.930190 12224 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 10:18:42.937582 12224 rpc_server.cc:307] RPC server started. Bound to: 127.0.206.66:38849
I20260619 10:18:42.938071 12224 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781864224577058-825-0/minicluster-data/ts-1/data/info.pb
I20260619 10:18:42.940590 12353 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.206.66:38849 every 8 connection(s)
I20260619 10:18:42.945773 825 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTU9OP6/build/release/bin/kudu as pid 12224
I20260619 10:18:42.945894 825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781864224577058-825-0/minicluster-data/ts-1/wal/instance
I20260619 10:18:42.946097 825 external_mini_cluster.cc:1428] Setting key c09361bd222d9108f8951f3ac847cde6
Jun 19 10:18:42 dist-test-slave-6pt2 krb5kdc[11528](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864322, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.66@KRBTEST.COM for kudu/127.0.206.126@KRBTEST.COM
I20260619 10:18:42.952046 12220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:18:42.941342 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.66:48747 (local address 127.0.206.126:35047)
0619 10:18:42.941388 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0619 10:18:42.941390 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 10:18:42.941582 (+ 192us) server_negotiation.cc:407] Connection header received
0619 10:18:42.942286 (+ 704us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:18:42.942288 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:18:42.942304 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:18:42.942336 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:18:42.943209 (+ 873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:18:42.943997 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:18:42.945476 (+ 1479us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:18:42.945774 (+ 298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:18:42.945821 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:18:42.948714 (+ 2893us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 10:18:42.948715 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 10:18:42.948715 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 10:18:42.948722 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 10:18:42.950435 (+ 1713us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:18:42.950980 (+ 545us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:18:42.950981 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:18:42.950982 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:18:42.951039 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:18:42.951311 (+ 272us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:18:42.951312 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:18:42.951312 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:18:42.951440 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 10:18:42.951493 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0619 10:18:42.951852 (+ 359us) server_negotiation.cc:299] Negotiation successful
0619 10:18:42.951953 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260619 10:18:42.952965 12356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:18:42.941185 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:35047 (local address 127.0.206.66:48747)
0619 10:18:42.941460 (+ 275us) negotiation.cc:107] Waiting for socket to connect
0619 10:18:42.941501 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0619 10:18:42.942155 (+ 654us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 10:18:42.942517 (+ 362us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 10:18:42.942521 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 10:18:42.942785 (+ 264us) client_negotiation.cc:189] Negotiated authn=SASL
0619 10:18:42.943093 (+ 308us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:18:42.943100 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:18:42.944112 (+ 1012us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:18:42.944113 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:18:42.945372 (+ 1259us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:18:42.945375 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:18:42.946019 (+ 644us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:18:42.946020 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:18:42.946223 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:18:42.946864 (+ 641us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 10:18:42.946875 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0619 10:18:42.948610 (+ 1735us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 10:18:42.950533 (+ 1923us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 10:18:42.950539 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 10:18:42.950542 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 10:18:42.950878 (+ 336us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 10:18:42.951104 (+ 226us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 10:18:42.951105 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 10:18:42.951105 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 10:18:42.951239 (+ 134us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 10:18:42.951525 (+ 286us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 10:18:42.951532 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 10:18:42.951692 (+ 160us) client_negotiation.cc:769] Sending connection context
0619 10:18:42.952534 (+ 842us) client_negotiation.cc:240] Negotiation successful
0619 10:18:42.952696 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":235,"thread_start_us":73,"threads_started":1}
I20260619 10:18:42.953670 12354 heartbeater.cc:347] Connected to a master server at 127.0.206.126:35047
I20260619 10:18:42.953765 12354 heartbeater.cc:464] Registering TS with master...
I20260619 10:18:42.953951 12354 heartbeater.cc:511] Master 127.0.206.126:35047 requested a full tablet report, sending...
I20260619 10:18:42.954372 12004 ts_manager.cc:194] Registered new tserver with Master: d7c17b58dde54337ba12510a19f60697 (127.0.206.66:38849)
I20260619 10:18:42.954905 12004 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.206.66@KRBTEST.COM'} at 127.0.206.66:48747
WARNING: no policy specified for kudu/127.0.206.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.206.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.206.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781864224577058-825-0/krb5kdc/kudu_127.0.206.67.keytab.
Entry for principal kudu/127.0.206.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781864224577058-825-0/krb5kdc/kudu_127.0.206.67.keytab.
I20260619 10:18:42.978580 825 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTU9OP6/build/release/bin/kudu
/tmp/dist-test-taskTU9OP6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781864224577058-825-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781864224577058-825-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781864224577058-825-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781864224577058-825-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.0.206.84:52435/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.206.67:0
--local_ip_for_outbound_sockets=127.0.206.67
--webserver_interface=127.0.206.67
--webserver_port=0
--tserver_master_addrs=127.0.206.126:35047
--keytab_file=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781864224577058-825-0/krb5kdc/kudu_127.0.206.67.keytab
--principal=kudu/127.0.206.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.206.84:43823
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781864224577058-825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781864224577058-825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781864224577058-825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 10:18:43.127228 12359 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 10:18:43.127597 12359 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 10:18:43.127682 12359 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 10:18:43.129721 12359 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 10:18:43.129822 12359 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 10:18:43.129904 12359 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.206.67
I20260619 10:18:43.132329 12359 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.206.84:43823
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781864224577058-825-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781864224577058-825-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.0.206.84:52435/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781864224577058-825-0/krb5kdc/kudu_127.0.206.67.keytab
--principal=kudu/127.0.206.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.206.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781864224577058-825-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.206.67
--webserver_port=0
--tserver_master_addrs=127.0.206.126:35047
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12359
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.206.67
--log_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781864224577058-825-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 09:32:44 UTC on ccycloud.quasar-gbtldp.root.comops.site
build id 4455440
I20260619 10:18:43.132798 12359 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 10:18:43.133121 12359 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 10:18:43.147881 12365 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 10:18:43.148335 12364 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 10:18:43.148567 12367 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 10:18:43.149330 12359 server_base.cc:1034] running on GCE node
I20260619 10:18:43.149540 12359 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 10:18:43.149788 12359 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 10:18:43.165227 12359 hybrid_clock.cc:648] HybridClock initialized: now 1781864323165195 us; error 57 us; skew 500 ppm
Jun 19 10:18:43 dist-test-slave-6pt2 krb5kdc[11528](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864323, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 10:18:43.172644 12359 init.cc:377] Logged in from keytab as kudu/127.0.206.67@KRBTEST.COM (short username kudu)
I20260619 10:18:43.173159 12359 webserver.cc:460] Webserver started at http://127.0.206.67:34269/ using document root <none> and password file <none>
I20260619 10:18:43.173408 12359 fs_manager.cc:359] Metadata directory not provided
I20260619 10:18:43.173453 12359 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 10:18:43.173571 12359 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Jun 19 10:18:43 dist-test-slave-6pt2 krb5kdc[11528](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864323, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.67@KRBTEST.COM for HTTP/127.0.206.84@KRBTEST.COM
I20260619 10:18:43.192350 12359 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781864224577058-825-0/minicluster-data/ts-2/data/instance:
uuid: "a6d4bbbd26b24bfab203c0fdbce9b4ec"
format_stamp: "Formatted at 2026-06-19 10:18:43 on dist-test-slave-6pt2"
server_key: "e50c4da24f8a4d738e4f735d70791d50"
server_key_iv: "bddf257f18c2ca119083b19dc5150995"
server_key_version: "kuduclusterkey@0"
I20260619 10:18:43.192806 12359 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781864224577058-825-0/minicluster-data/ts-2/wal/instance:
uuid: "a6d4bbbd26b24bfab203c0fdbce9b4ec"
format_stamp: "Formatted at 2026-06-19 10:18:43 on dist-test-slave-6pt2"
server_key: "e50c4da24f8a4d738e4f735d70791d50"
server_key_iv: "bddf257f18c2ca119083b19dc5150995"
server_key_version: "kuduclusterkey@0"
I20260619 10:18:43.211107 12359 fs_manager.cc:691] Time spent creating directory manager: real 0.018s user 0.004s sys 0.000s
I20260619 10:18:43.232929 12374 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 10:18:43.233189 12359 fs_manager.cc:725] Time spent opening block manager: real 0.010s user 0.000s sys 0.000s
I20260619 10:18:43.233284 12359 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781864224577058-825-0/minicluster-data/ts-2/data,/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781864224577058-825-0/minicluster-data/ts-2/wal
uuid: "a6d4bbbd26b24bfab203c0fdbce9b4ec"
format_stamp: "Formatted at 2026-06-19 10:18:43 on dist-test-slave-6pt2"
server_key: "e50c4da24f8a4d738e4f735d70791d50"
server_key_iv: "bddf257f18c2ca119083b19dc5150995"
server_key_version: "kuduclusterkey@0"
I20260619 10:18:43.233392 12359 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781864224577058-825-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781864224577058-825-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781864224577058-825-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 10:18:43.278398 12359 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 10:18:43.280866 12359 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 10:18:43.281069 12359 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 10:18:43.281347 12359 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 10:18:43.281879 12359 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 10:18:43.281973 12359 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 10:18:43.282042 12359 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 10:18:43.282095 12359 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 10:18:43.291803 12359 rpc_server.cc:307] RPC server started. Bound to: 127.0.206.67:42949
I20260619 10:18:43.292254 12359 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781864224577058-825-0/minicluster-data/ts-2/data/info.pb
Jun 19 10:18:43 dist-test-slave-6pt2 krb5kdc[11528](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864322, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.126@KRBTEST.COM for HTTP/127.0.206.84@KRBTEST.COM
I20260619 10:18:43.297241 12487 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.206.67:42949 every 8 connection(s)
I20260619 10:18:43.300734 825 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTU9OP6/build/release/bin/kudu as pid 12359
I20260619 10:18:43.300823 825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781864224577058-825-0/minicluster-data/ts-2/wal/instance
I20260619 10:18:43.301010 825 external_mini_cluster.cc:1428] Setting key cf26678865a06759a46559775a53377a
Jun 19 10:18:43 dist-test-slave-6pt2 krb5kdc[11528](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864323, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.67@KRBTEST.COM for kudu/127.0.206.126@KRBTEST.COM
I20260619 10:18:43.309770 12220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:18:43.297510 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.67:41487 (local address 127.0.206.126:35047)
0619 10:18:43.297555 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0619 10:18:43.297557 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 10:18:43.298854 (+ 1297us) server_negotiation.cc:407] Connection header received
0619 10:18:43.299528 (+ 674us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:18:43.299530 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:18:43.299543 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:18:43.299572 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:18:43.300369 (+ 797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:18:43.301182 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:18:43.302720 (+ 1538us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:18:43.302988 (+ 268us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:18:43.303028 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:18:43.305492 (+ 2464us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 10:18:43.305493 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 10:18:43.305494 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 10:18:43.305501 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 10:18:43.308386 (+ 2885us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:18:43.308815 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:18:43.308816 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:18:43.308816 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:18:43.308869 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:18:43.309113 (+ 244us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:18:43.309114 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:18:43.309115 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:18:43.309231 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 10:18:43.309268 (+ 37us) server_negotiation.cc:1035] Waiting for connection context
0619 10:18:43.309606 (+ 338us) server_negotiation.cc:299] Negotiation successful
0619 10:18:43.309685 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260619 10:18:43.310206 12490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:18:43.297492 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:35047 (local address 127.0.206.67:41487)
0619 10:18:43.298717 (+ 1225us) negotiation.cc:107] Waiting for socket to connect
0619 10:18:43.298766 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0619 10:18:43.299408 (+ 642us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 10:18:43.299753 (+ 345us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 10:18:43.299757 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 10:18:43.299986 (+ 229us) client_negotiation.cc:189] Negotiated authn=SASL
0619 10:18:43.300257 (+ 271us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:18:43.300262 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:18:43.301399 (+ 1137us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:18:43.301400 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:18:43.302624 (+ 1224us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:18:43.302627 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:18:43.303113 (+ 486us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:18:43.303114 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:18:43.303247 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:18:43.303833 (+ 586us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 10:18:43.303842 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0619 10:18:43.305391 (+ 1549us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 10:18:43.308474 (+ 3083us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 10:18:43.308480 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 10:18:43.308482 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 10:18:43.308740 (+ 258us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 10:18:43.308935 (+ 195us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 10:18:43.308935 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 10:18:43.308935 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 10:18:43.309051 (+ 116us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 10:18:43.309308 (+ 257us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 10:18:43.309313 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 10:18:43.309472 (+ 159us) client_negotiation.cc:769] Sending connection context
0619 10:18:43.309858 (+ 386us) client_negotiation.cc:240] Negotiation successful
0619 10:18:43.310001 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1176,"thread_start_us":118,"threads_started":1}
I20260619 10:18:43.314630 12488 heartbeater.cc:347] Connected to a master server at 127.0.206.126:35047
I20260619 10:18:43.314776 12488 heartbeater.cc:464] Registering TS with master...
I20260619 10:18:43.315006 12488 heartbeater.cc:511] Master 127.0.206.126:35047 requested a full tablet report, sending...
I20260619 10:18:43.315402 12004 ts_manager.cc:194] Registered new tserver with Master: a6d4bbbd26b24bfab203c0fdbce9b4ec (127.0.206.67:42949)
I20260619 10:18:43.315876 12004 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.206.67@KRBTEST.COM'} at 127.0.206.67:41487
I20260619 10:18:43.316504 825 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 10:18:43.334343 12220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:18:43.318027 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56674 (local address 127.0.206.126:35047)
0619 10:18:43.318063 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0619 10:18:43.318065 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 10:18:43.326827 (+ 8762us) server_negotiation.cc:407] Connection header received
0619 10:18:43.326942 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:18:43.326943 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:18:43.326968 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:18:43.326996 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:18:43.327967 (+ 971us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:18:43.328738 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:18:43.329875 (+ 1137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:18:43.330134 (+ 259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:18:43.330171 (+ 37us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:18:43.331719 (+ 1548us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 10:18:43.331720 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 10:18:43.331721 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 10:18:43.331727 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 10:18:43.333242 (+ 1515us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:18:43.333585 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:18:43.333586 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:18:43.333586 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:18:43.333632 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:18:43.333817 (+ 185us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:18:43.333818 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:18:43.333818 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:18:43.333928 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 10:18:43.333993 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0619 10:18:43.334183 (+ 190us) server_negotiation.cc:299] Negotiation successful
0619 10:18:43.334254 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260619 10:18:43.336344 12004 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56674:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 10:18:43.338513 12004 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 10:18:43.348323 12503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:18:43.342054 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.66:38849 (local address 127.0.0.1:33884)
0619 10:18:43.343511 (+ 1457us) negotiation.cc:107] Waiting for socket to connect
0619 10:18:43.343517 (+ 6us) client_negotiation.cc:174] Beginning negotiation
0619 10:18:43.343556 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 10:18:43.343810 (+ 254us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 10:18:43.343813 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 10:18:43.343815 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 10:18:43.343872 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:18:43.343873 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:18:43.345382 (+ 1509us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:18:43.345383 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:18:43.347021 (+ 1638us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:18:43.347024 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:18:43.348052 (+ 1028us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:18:43.348053 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:18:43.348128 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:18:43.348140 (+ 12us) client_negotiation.cc:769] Sending connection context
0619 10:18:43.348202 (+ 62us) client_negotiation.cc:240] Negotiation successful
0619 10:18:43.348250 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1447,"thread_start_us":48,"threads_started":1}
I20260619 10:18:43.348927 12506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:18:43.342235 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33884 (local address 127.0.206.66:38849)
0619 10:18:43.343338 (+ 1103us) server_negotiation.cc:206] Beginning negotiation
0619 10:18:43.343343 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 10:18:43.343596 (+ 253us) server_negotiation.cc:407] Connection header received
0619 10:18:43.343632 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:18:43.343634 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:18:43.343647 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:18:43.343699 (+ 52us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 10:18:43.343941 (+ 242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:18:43.345244 (+ 1303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:18:43.347119 (+ 1875us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:18:43.347871 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:18:43.347919 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:18:43.347984 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0619 10:18:43.348746 (+ 762us) server_negotiation.cc:299] Negotiation successful
0619 10:18:43.348818 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1064,"thread_start_us":61,"threads_started":1}
I20260619 10:18:43.350122 12288 tablet_service.cc:1467] Processing CreateTablet for tablet 7b8505d957f0476aaca23896ec9c99bf (DEFAULT_TABLE table=test-table [id=91ff76a6ce4d4b98aadc625b72846a2d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 10:18:43.350476 12288 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7b8505d957f0476aaca23896ec9c99bf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 10:18:43.361570 12507 tablet_bootstrap.cc:492] T 7b8505d957f0476aaca23896ec9c99bf P d7c17b58dde54337ba12510a19f60697: Bootstrap starting.
I20260619 10:18:43.364521 12507 tablet_bootstrap.cc:654] T 7b8505d957f0476aaca23896ec9c99bf P d7c17b58dde54337ba12510a19f60697: Neither blocks nor log segments found. Creating new log.
I20260619 10:18:43.364897 12505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:18:43.341692 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.65:34117 (local address 127.0.0.1:47776)
0619 10:18:43.342664 (+ 972us) negotiation.cc:107] Waiting for socket to connect
0619 10:18:43.342702 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0619 10:18:43.343202 (+ 500us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 10:18:43.355156 (+ 11954us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 10:18:43.355158 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 10:18:43.355160 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 10:18:43.355200 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:18:43.355201 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:18:43.358840 (+ 3639us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:18:43.358841 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:18:43.359983 (+ 1142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:18:43.359985 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:18:43.364686 (+ 4701us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:18:43.364687 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:18:43.364748 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:18:43.364752 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 10:18:43.364782 (+ 30us) client_negotiation.cc:240] Negotiation successful
0619 10:18:43.364828 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":944,"thread_start_us":64,"threads_started":1}
I20260619 10:18:43.365478 12502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:18:43.341790 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47776 (local address 127.0.206.65:34117)
0619 10:18:43.354804 (+ 13014us) server_negotiation.cc:206] Beginning negotiation
0619 10:18:43.354809 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 10:18:43.354822 (+ 13us) server_negotiation.cc:407] Connection header received
0619 10:18:43.354856 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:18:43.354858 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:18:43.354875 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:18:43.355263 (+ 388us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 10:18:43.355325 (+ 62us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:18:43.356283 (+ 958us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:18:43.360080 (+ 3797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:18:43.360812 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:18:43.365228 (+ 4416us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:18:43.365303 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0619 10:18:43.365329 (+ 26us) server_negotiation.cc:299] Negotiation successful
0619 10:18:43.365413 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12964,"thread_start_us":54,"threads_started":1}
I20260619 10:18:43.366222 12150 tablet_service.cc:1467] Processing CreateTablet for tablet 7b8505d957f0476aaca23896ec9c99bf (DEFAULT_TABLE table=test-table [id=91ff76a6ce4d4b98aadc625b72846a2d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 10:18:43.366539 12150 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7b8505d957f0476aaca23896ec9c99bf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 10:18:43.369989 12501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:18:43.341903 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.67:42949 (local address 127.0.0.1:39116)
0619 10:18:43.343380 (+ 1477us) negotiation.cc:107] Waiting for socket to connect
0619 10:18:43.343388 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0619 10:18:43.343417 (+ 29us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 10:18:43.356506 (+ 13089us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 10:18:43.356507 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 10:18:43.356509 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 10:18:43.356547 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:18:43.356548 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:18:43.361783 (+ 5235us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:18:43.361784 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:18:43.362840 (+ 1056us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:18:43.362842 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:18:43.369764 (+ 6922us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:18:43.369765 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:18:43.369843 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:18:43.369847 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 10:18:43.369874 (+ 27us) client_negotiation.cc:240] Negotiation successful
0619 10:18:43.369925 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1464,"thread_start_us":63,"threads_started":1}
I20260619 10:18:43.370728 12504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:18:43.341999 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39116 (local address 127.0.206.67:42949)
0619 10:18:43.355011 (+ 13012us) server_negotiation.cc:206] Beginning negotiation
0619 10:18:43.355015 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 10:18:43.355022 (+ 7us) server_negotiation.cc:407] Connection header received
0619 10:18:43.355052 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:18:43.355054 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:18:43.355067 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:18:43.356625 (+ 1558us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 10:18:43.356702 (+ 77us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:18:43.361078 (+ 4376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:18:43.362967 (+ 1889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:18:43.369681 (+ 6714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:18:43.370480 (+ 799us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:18:43.370565 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0619 10:18:43.370589 (+ 24us) server_negotiation.cc:299] Negotiation successful
0619 10:18:43.370657 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12981,"thread_start_us":98,"threads_started":1}
I20260619 10:18:43.371183 12507 log.cc:824] T 7b8505d957f0476aaca23896ec9c99bf P d7c17b58dde54337ba12510a19f60697: Log is configured to *not* fsync() on all Append() calls
I20260619 10:18:43.371861 12417 tablet_service.cc:1467] Processing CreateTablet for tablet 7b8505d957f0476aaca23896ec9c99bf (DEFAULT_TABLE table=test-table [id=91ff76a6ce4d4b98aadc625b72846a2d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 10:18:43.372171 12417 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7b8505d957f0476aaca23896ec9c99bf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 10:18:43.376017 12508 tablet_bootstrap.cc:492] T 7b8505d957f0476aaca23896ec9c99bf P b461caa3dc3741278f936964a3edfe88: Bootstrap starting.
I20260619 10:18:43.390930 12508 tablet_bootstrap.cc:654] T 7b8505d957f0476aaca23896ec9c99bf P b461caa3dc3741278f936964a3edfe88: Neither blocks nor log segments found. Creating new log.
I20260619 10:18:43.391336 12508 log.cc:824] T 7b8505d957f0476aaca23896ec9c99bf P b461caa3dc3741278f936964a3edfe88: Log is configured to *not* fsync() on all Append() calls
I20260619 10:18:43.391578 12509 tablet_bootstrap.cc:492] T 7b8505d957f0476aaca23896ec9c99bf P a6d4bbbd26b24bfab203c0fdbce9b4ec: Bootstrap starting.
I20260619 10:18:43.392799 12509 tablet_bootstrap.cc:654] T 7b8505d957f0476aaca23896ec9c99bf P a6d4bbbd26b24bfab203c0fdbce9b4ec: Neither blocks nor log segments found. Creating new log.
I20260619 10:18:43.393638 12509 log.cc:824] T 7b8505d957f0476aaca23896ec9c99bf P a6d4bbbd26b24bfab203c0fdbce9b4ec: Log is configured to *not* fsync() on all Append() calls
I20260619 10:18:43.394239 12507 tablet_bootstrap.cc:492] T 7b8505d957f0476aaca23896ec9c99bf P d7c17b58dde54337ba12510a19f60697: No bootstrap required, opened a new log
I20260619 10:18:43.394361 12507 ts_tablet_manager.cc:1397] T 7b8505d957f0476aaca23896ec9c99bf P d7c17b58dde54337ba12510a19f60697: Time spent bootstrapping tablet: real 0.033s user 0.002s sys 0.000s
I20260619 10:18:43.396978 12507 raft_consensus.cc:348] T 7b8505d957f0476aaca23896ec9c99bf P d7c17b58dde54337ba12510a19f60697 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b461caa3dc3741278f936964a3edfe88" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 34117 } } peers { permanent_uuid: "d7c17b58dde54337ba12510a19f60697" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 38849 } } peers { permanent_uuid: "a6d4bbbd26b24bfab203c0fdbce9b4ec" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 42949 } }
I20260619 10:18:43.397162 12507 raft_consensus.cc:374] T 7b8505d957f0476aaca23896ec9c99bf P d7c17b58dde54337ba12510a19f60697 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 10:18:43.397209 12507 raft_consensus.cc:729] T 7b8505d957f0476aaca23896ec9c99bf P d7c17b58dde54337ba12510a19f60697 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7c17b58dde54337ba12510a19f60697, State: Initialized, Role: FOLLOWER
I20260619 10:18:43.397390 12507 consensus_queue.cc:260] T 7b8505d957f0476aaca23896ec9c99bf P d7c17b58dde54337ba12510a19f60697 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b461caa3dc3741278f936964a3edfe88" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 34117 } } peers { permanent_uuid: "d7c17b58dde54337ba12510a19f60697" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 38849 } } peers { permanent_uuid: "a6d4bbbd26b24bfab203c0fdbce9b4ec" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 42949 } }
I20260619 10:18:43.397651 12507 ts_tablet_manager.cc:1428] T 7b8505d957f0476aaca23896ec9c99bf P d7c17b58dde54337ba12510a19f60697: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260619 10:18:43.397812 12354 heartbeater.cc:503] Master 127.0.206.126:35047 was elected leader, sending a full tablet report...
I20260619 10:18:43.403275 12508 tablet_bootstrap.cc:492] T 7b8505d957f0476aaca23896ec9c99bf P b461caa3dc3741278f936964a3edfe88: No bootstrap required, opened a new log
I20260619 10:18:43.403416 12508 ts_tablet_manager.cc:1397] T 7b8505d957f0476aaca23896ec9c99bf P b461caa3dc3741278f936964a3edfe88: Time spent bootstrapping tablet: real 0.027s user 0.002s sys 0.000s
I20260619 10:18:43.403797 12509 tablet_bootstrap.cc:492] T 7b8505d957f0476aaca23896ec9c99bf P a6d4bbbd26b24bfab203c0fdbce9b4ec: No bootstrap required, opened a new log
I20260619 10:18:43.403877 12509 ts_tablet_manager.cc:1397] T 7b8505d957f0476aaca23896ec9c99bf P a6d4bbbd26b24bfab203c0fdbce9b4ec: Time spent bootstrapping tablet: real 0.012s user 0.002s sys 0.000s
I20260619 10:18:43.405808 12509 raft_consensus.cc:348] T 7b8505d957f0476aaca23896ec9c99bf P a6d4bbbd26b24bfab203c0fdbce9b4ec [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b461caa3dc3741278f936964a3edfe88" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 34117 } } peers { permanent_uuid: "d7c17b58dde54337ba12510a19f60697" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 38849 } } peers { permanent_uuid: "a6d4bbbd26b24bfab203c0fdbce9b4ec" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 42949 } }
I20260619 10:18:43.406014 12509 raft_consensus.cc:374] T 7b8505d957f0476aaca23896ec9c99bf P a6d4bbbd26b24bfab203c0fdbce9b4ec [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 10:18:43.406062 12509 raft_consensus.cc:729] T 7b8505d957f0476aaca23896ec9c99bf P a6d4bbbd26b24bfab203c0fdbce9b4ec [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a6d4bbbd26b24bfab203c0fdbce9b4ec, State: Initialized, Role: FOLLOWER
I20260619 10:18:43.406193 12509 consensus_queue.cc:260] T 7b8505d957f0476aaca23896ec9c99bf P a6d4bbbd26b24bfab203c0fdbce9b4ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b461caa3dc3741278f936964a3edfe88" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 34117 } } peers { permanent_uuid: "d7c17b58dde54337ba12510a19f60697" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 38849 } } peers { permanent_uuid: "a6d4bbbd26b24bfab203c0fdbce9b4ec" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 42949 } }
I20260619 10:18:43.406461 12509 ts_tablet_manager.cc:1428] T 7b8505d957f0476aaca23896ec9c99bf P a6d4bbbd26b24bfab203c0fdbce9b4ec: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260619 10:18:43.406592 12488 heartbeater.cc:503] Master 127.0.206.126:35047 was elected leader, sending a full tablet report...
I20260619 10:18:43.406648 12508 raft_consensus.cc:348] T 7b8505d957f0476aaca23896ec9c99bf P b461caa3dc3741278f936964a3edfe88 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b461caa3dc3741278f936964a3edfe88" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 34117 } } peers { permanent_uuid: "d7c17b58dde54337ba12510a19f60697" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 38849 } } peers { permanent_uuid: "a6d4bbbd26b24bfab203c0fdbce9b4ec" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 42949 } }
I20260619 10:18:43.407272 12508 raft_consensus.cc:374] T 7b8505d957f0476aaca23896ec9c99bf P b461caa3dc3741278f936964a3edfe88 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 10:18:43.407354 12508 raft_consensus.cc:729] T 7b8505d957f0476aaca23896ec9c99bf P b461caa3dc3741278f936964a3edfe88 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b461caa3dc3741278f936964a3edfe88, State: Initialized, Role: FOLLOWER
I20260619 10:18:43.407517 12508 consensus_queue.cc:260] T 7b8505d957f0476aaca23896ec9c99bf P b461caa3dc3741278f936964a3edfe88 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b461caa3dc3741278f936964a3edfe88" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 34117 } } peers { permanent_uuid: "d7c17b58dde54337ba12510a19f60697" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 38849 } } peers { permanent_uuid: "a6d4bbbd26b24bfab203c0fdbce9b4ec" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 42949 } }
I20260619 10:18:43.407819 12508 ts_tablet_manager.cc:1428] T 7b8505d957f0476aaca23896ec9c99bf P b461caa3dc3741278f936964a3edfe88: Time spent starting tablet: real 0.004s user 0.003s sys 0.000s
I20260619 10:18:43.407958 12216 heartbeater.cc:503] Master 127.0.206.126:35047 was elected leader, sending a full tablet report...
W20260619 10:18:43.410040 12217 tablet.cc:2369] T 7b8505d957f0476aaca23896ec9c99bf P b461caa3dc3741278f936964a3edfe88: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260619 10:18:43.441365 12355 tablet.cc:2369] T 7b8505d957f0476aaca23896ec9c99bf P d7c17b58dde54337ba12510a19f60697: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260619 10:18:43.547180 12489 tablet.cc:2369] T 7b8505d957f0476aaca23896ec9c99bf P a6d4bbbd26b24bfab203c0fdbce9b4ec: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 10:18:43.552639 12515 raft_consensus.cc:482] T 7b8505d957f0476aaca23896ec9c99bf P b461caa3dc3741278f936964a3edfe88 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 10:18:43.552804 12515 raft_consensus.cc:504] T 7b8505d957f0476aaca23896ec9c99bf P b461caa3dc3741278f936964a3edfe88 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b461caa3dc3741278f936964a3edfe88" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 34117 } } peers { permanent_uuid: "d7c17b58dde54337ba12510a19f60697" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 38849 } } peers { permanent_uuid: "a6d4bbbd26b24bfab203c0fdbce9b4ec" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 42949 } }
I20260619 10:18:43.553192 12515 leader_election.cc:288] T 7b8505d957f0476aaca23896ec9c99bf P b461caa3dc3741278f936964a3edfe88 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d7c17b58dde54337ba12510a19f60697 (127.0.206.66:38849), a6d4bbbd26b24bfab203c0fdbce9b4ec (127.0.206.67:42949)
I20260619 10:18:43.557351 12512 raft_consensus.cc:482] T 7b8505d957f0476aaca23896ec9c99bf P d7c17b58dde54337ba12510a19f60697 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 10:18:43.557498 12516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:18:43.553393 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.66:38849 (local address 127.0.206.65:56285)
0619 10:18:43.553627 (+ 234us) negotiation.cc:107] Waiting for socket to connect
0619 10:18:43.553638 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 10:18:43.553693 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 10:18:43.554016 (+ 323us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 10:18:43.554017 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 10:18:43.554019 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 10:18:43.554116 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:18:43.554118 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:18:43.555038 (+ 920us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:18:43.555039 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:18:43.556280 (+ 1241us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:18:43.556283 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:18:43.557095 (+ 812us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:18:43.557095 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:18:43.557182 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:18:43.557186 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 10:18:43.557369 (+ 183us) client_negotiation.cc:240] Negotiation successful
0619 10:18:43.557414 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":205,"mutex_wait_us":41,"thread_start_us":89,"threads_started":1}
I20260619 10:18:43.557463 12512 raft_consensus.cc:504] T 7b8505d957f0476aaca23896ec9c99bf P d7c17b58dde54337ba12510a19f60697 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b461caa3dc3741278f936964a3edfe88" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 34117 } } peers { permanent_uuid: "d7c17b58dde54337ba12510a19f60697" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 38849 } } peers { permanent_uuid: "a6d4bbbd26b24bfab203c0fdbce9b4ec" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 42949 } }
I20260619 10:18:43.557633 12504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:18:43.553716 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.65:35731 (local address 127.0.206.67:42949)
0619 10:18:43.553761 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0619 10:18:43.553763 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 10:18:43.553883 (+ 120us) server_negotiation.cc:407] Connection header received
0619 10:18:43.553979 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:18:43.553980 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:18:43.553988 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:18:43.554016 (+ 28us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 10:18:43.554236 (+ 220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:18:43.555028 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:18:43.556586 (+ 1558us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:18:43.557159 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:18:43.557209 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:18:43.557217 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0619 10:18:43.557526 (+ 309us) server_negotiation.cc:299] Negotiation successful
0619 10:18:43.557566 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260619 10:18:43.557840 12506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:18:43.553816 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.65:56285 (local address 127.0.206.66:38849)
0619 10:18:43.553865 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0619 10:18:43.553867 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 10:18:43.553876 (+ 9us) server_negotiation.cc:407] Connection header received
0619 10:18:43.553903 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:18:43.553904 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:18:43.553914 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:18:43.553963 (+ 49us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 10:18:43.554191 (+ 228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:18:43.554938 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:18:43.556394 (+ 1456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:18:43.556970 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:18:43.557022 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:18:43.557034 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0619 10:18:43.557273 (+ 239us) server_negotiation.cc:299] Negotiation successful
0619 10:18:43.557313 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260619 10:18:43.558050 12305 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7b8505d957f0476aaca23896ec9c99bf" candidate_uuid: "b461caa3dc3741278f936964a3edfe88" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7c17b58dde54337ba12510a19f60697" is_pre_election: true
I20260619 10:18:43.558142 12305 raft_consensus.cc:2454] T 7b8505d957f0476aaca23896ec9c99bf P d7c17b58dde54337ba12510a19f60697 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b461caa3dc3741278f936964a3edfe88 in term 0.
I20260619 10:18:43.558406 12105 leader_election.cc:302] T 7b8505d957f0476aaca23896ec9c99bf P b461caa3dc3741278f936964a3edfe88 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b461caa3dc3741278f936964a3edfe88, d7c17b58dde54337ba12510a19f60697; no voters:
I20260619 10:18:43.558521 12517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:18:43.553506 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.67:42949 (local address 127.0.206.65:35731)
0619 10:18:43.553823 (+ 317us) negotiation.cc:107] Waiting for socket to connect
0619 10:18:43.553833 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 10:18:43.553905 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 10:18:43.554078 (+ 173us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 10:18:43.554078 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 10:18:43.554080 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 10:18:43.554163 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:18:43.554165 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:18:43.555244 (+ 1079us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:18:43.555245 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:18:43.556480 (+ 1235us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:18:43.556482 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:18:43.557358 (+ 876us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:18:43.557359 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:18:43.557440 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:18:43.557443 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 10:18:43.558372 (+ 929us) client_negotiation.cc:240] Negotiation successful
0619 10:18:43.558414 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":302,"mutex_wait_us":2,"thread_start_us":94,"threads_started":1}
I20260619 10:18:43.558542 12515 raft_consensus.cc:2781] T 7b8505d957f0476aaca23896ec9c99bf P b461caa3dc3741278f936964a3edfe88 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 10:18:43.558584 12515 raft_consensus.cc:482] T 7b8505d957f0476aaca23896ec9c99bf P b461caa3dc3741278f936964a3edfe88 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 10:18:43.558631 12515 raft_consensus.cc:3037] T 7b8505d957f0476aaca23896ec9c99bf P b461caa3dc3741278f936964a3edfe88 [term 0 FOLLOWER]: Advancing to term 1
I20260619 10:18:43.558821 12442 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7b8505d957f0476aaca23896ec9c99bf" candidate_uuid: "b461caa3dc3741278f936964a3edfe88" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a6d4bbbd26b24bfab203c0fdbce9b4ec" is_pre_election: true
I20260619 10:18:43.558951 12442 raft_consensus.cc:2454] T 7b8505d957f0476aaca23896ec9c99bf P a6d4bbbd26b24bfab203c0fdbce9b4ec [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b461caa3dc3741278f936964a3edfe88 in term 0.
I20260619 10:18:43.559527 12515 raft_consensus.cc:504] T 7b8505d957f0476aaca23896ec9c99bf P b461caa3dc3741278f936964a3edfe88 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b461caa3dc3741278f936964a3edfe88" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 34117 } } peers { permanent_uuid: "d7c17b58dde54337ba12510a19f60697" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 38849 } } peers { permanent_uuid: "a6d4bbbd26b24bfab203c0fdbce9b4ec" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 42949 } }
I20260619 10:18:43.559650 12515 leader_election.cc:288] T 7b8505d957f0476aaca23896ec9c99bf P b461caa3dc3741278f936964a3edfe88 [CANDIDATE]: Term 1 election: Requested vote from peers d7c17b58dde54337ba12510a19f60697 (127.0.206.66:38849), a6d4bbbd26b24bfab203c0fdbce9b4ec (127.0.206.67:42949)
I20260619 10:18:43.560021 12305 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7b8505d957f0476aaca23896ec9c99bf" candidate_uuid: "b461caa3dc3741278f936964a3edfe88" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7c17b58dde54337ba12510a19f60697"
I20260619 10:18:43.560092 12305 raft_consensus.cc:3037] T 7b8505d957f0476aaca23896ec9c99bf P d7c17b58dde54337ba12510a19f60697 [term 0 FOLLOWER]: Advancing to term 1
I20260619 10:18:43.560745 12442 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7b8505d957f0476aaca23896ec9c99bf" candidate_uuid: "b461caa3dc3741278f936964a3edfe88" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a6d4bbbd26b24bfab203c0fdbce9b4ec"
I20260619 10:18:43.560817 12442 raft_consensus.cc:3037] T 7b8505d957f0476aaca23896ec9c99bf P a6d4bbbd26b24bfab203c0fdbce9b4ec [term 0 FOLLOWER]: Advancing to term 1
I20260619 10:18:43.560913 12305 raft_consensus.cc:2454] T 7b8505d957f0476aaca23896ec9c99bf P d7c17b58dde54337ba12510a19f60697 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b461caa3dc3741278f936964a3edfe88 in term 1.
I20260619 10:18:43.561112 12105 leader_election.cc:302] T 7b8505d957f0476aaca23896ec9c99bf P b461caa3dc3741278f936964a3edfe88 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b461caa3dc3741278f936964a3edfe88, d7c17b58dde54337ba12510a19f60697; no voters:
I20260619 10:18:43.557806 12512 leader_election.cc:288] T 7b8505d957f0476aaca23896ec9c99bf P d7c17b58dde54337ba12510a19f60697 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b461caa3dc3741278f936964a3edfe88 (127.0.206.65:34117), a6d4bbbd26b24bfab203c0fdbce9b4ec (127.0.206.67:42949)
I20260619 10:18:43.561280 12515 raft_consensus.cc:2781] T 7b8505d957f0476aaca23896ec9c99bf P b461caa3dc3741278f936964a3edfe88 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 10:18:43.561405 12515 raft_consensus.cc:686] T 7b8505d957f0476aaca23896ec9c99bf P b461caa3dc3741278f936964a3edfe88 [term 1 LEADER]: Becoming Leader. State: Replica: b461caa3dc3741278f936964a3edfe88, State: Running, Role: LEADER
I20260619 10:18:43.561486 12515 consensus_queue.cc:237] T 7b8505d957f0476aaca23896ec9c99bf P b461caa3dc3741278f936964a3edfe88 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b461caa3dc3741278f936964a3edfe88" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 34117 } } peers { permanent_uuid: "d7c17b58dde54337ba12510a19f60697" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 38849 } } peers { permanent_uuid: "a6d4bbbd26b24bfab203c0fdbce9b4ec" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 42949 } }
I20260619 10:18:43.561647 12442 raft_consensus.cc:2454] T 7b8505d957f0476aaca23896ec9c99bf P a6d4bbbd26b24bfab203c0fdbce9b4ec [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b461caa3dc3741278f936964a3edfe88 in term 1.
I20260619 10:18:43.562314 12004 catalog_manager.cc:5495] T 7b8505d957f0476aaca23896ec9c99bf P b461caa3dc3741278f936964a3edfe88 reported cstate change: term changed from 0 to 1, leader changed from <none> to b461caa3dc3741278f936964a3edfe88 (127.0.206.65). New cstate: current_term: 1 leader_uuid: "b461caa3dc3741278f936964a3edfe88" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b461caa3dc3741278f936964a3edfe88" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 34117 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d7c17b58dde54337ba12510a19f60697" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 38849 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a6d4bbbd26b24bfab203c0fdbce9b4ec" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 42949 } health_report { overall_health: UNKNOWN } } }
I20260619 10:18:43.564894 12502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:18:43.558230 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.66:38623 (local address 127.0.206.65:34117)
0619 10:18:43.558280 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0619 10:18:43.558282 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 10:18:43.559262 (+ 980us) server_negotiation.cc:407] Connection header received
0619 10:18:43.559368 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:18:43.559368 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:18:43.559377 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:18:43.559403 (+ 26us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 10:18:43.559584 (+ 181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:18:43.560229 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:18:43.564001 (+ 3772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:18:43.564518 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:18:43.564567 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:18:43.564574 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0619 10:18:43.564787 (+ 213us) server_negotiation.cc:299] Negotiation successful
0619 10:18:43.564835 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260619 10:18:43.565241 12518 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:18:43.558029 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.65:34117 (local address 127.0.206.66:38623)
0619 10:18:43.559204 (+ 1175us) negotiation.cc:107] Waiting for socket to connect
0619 10:18:43.559214 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 10:18:43.559284 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 10:18:43.559454 (+ 170us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 10:18:43.559455 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 10:18:43.559456 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 10:18:43.559519 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:18:43.559520 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:18:43.562729 (+ 3209us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:18:43.562730 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:18:43.563898 (+ 1168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:18:43.563901 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:18:43.564634 (+ 733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:18:43.564635 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:18:43.564711 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:18:43.564715 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 10:18:43.565107 (+ 392us) client_negotiation.cc:240] Negotiation successful
0619 10:18:43.565162 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1160,"thread_start_us":61,"threads_started":1}
I20260619 10:18:43.565655 12170 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7b8505d957f0476aaca23896ec9c99bf" candidate_uuid: "d7c17b58dde54337ba12510a19f60697" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b461caa3dc3741278f936964a3edfe88" is_pre_election: true
I20260619 10:18:43.566524 12519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:18:43.558175 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.67:42949 (local address 127.0.206.66:33407)
0619 10:18:43.561961 (+ 3786us) negotiation.cc:107] Waiting for socket to connect
0619 10:18:43.561972 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 10:18:43.562029 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 10:18:43.563391 (+ 1362us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 10:18:43.563392 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 10:18:43.563408 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 10:18:43.563471 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:18:43.563472 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:18:43.564326 (+ 854us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:18:43.564327 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:18:43.565547 (+ 1220us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:18:43.565549 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:18:43.566280 (+ 731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:18:43.566281 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:18:43.566359 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:18:43.566362 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 10:18:43.566415 (+ 53us) client_negotiation.cc:240] Negotiation successful
0619 10:18:43.566448 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3768,"thread_start_us":63,"threads_started":1}
I20260619 10:18:43.566573 12504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:18:43.558326 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.66:33407 (local address 127.0.206.67:42949)
0619 10:18:43.558358 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0619 10:18:43.558359 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 10:18:43.562034 (+ 3675us) server_negotiation.cc:407] Connection header received
0619 10:18:43.563021 (+ 987us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:18:43.563022 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:18:43.563033 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:18:43.563091 (+ 58us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 10:18:43.563553 (+ 462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:18:43.564218 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:18:43.565627 (+ 1409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:18:43.566152 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:18:43.566199 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:18:43.566205 (+ 6us) server_negotiation.cc:1035] Waiting for connection context
0619 10:18:43.566422 (+ 217us) server_negotiation.cc:299] Negotiation successful
0619 10:18:43.566491 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260619 10:18:43.567034 12442 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7b8505d957f0476aaca23896ec9c99bf" candidate_uuid: "d7c17b58dde54337ba12510a19f60697" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a6d4bbbd26b24bfab203c0fdbce9b4ec" is_pre_election: true
I20260619 10:18:43.567139 12442 raft_consensus.cc:2379] T 7b8505d957f0476aaca23896ec9c99bf P a6d4bbbd26b24bfab203c0fdbce9b4ec [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d7c17b58dde54337ba12510a19f60697 in current term 1: Already voted for candidate b461caa3dc3741278f936964a3edfe88 in this term.
I20260619 10:18:43.568018 12242 leader_election.cc:302] T 7b8505d957f0476aaca23896ec9c99bf P d7c17b58dde54337ba12510a19f60697 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d7c17b58dde54337ba12510a19f60697; no voters: a6d4bbbd26b24bfab203c0fdbce9b4ec, b461caa3dc3741278f936964a3edfe88
I20260619 10:18:43.568197 12512 raft_consensus.cc:2726] T 7b8505d957f0476aaca23896ec9c99bf P d7c17b58dde54337ba12510a19f60697 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260619 10:18:43.577697 12502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:18:43.574458 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47790 (local address 127.0.206.65:34117)
0619 10:18:43.574523 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0619 10:18:43.574525 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 10:18:43.574534 (+ 9us) server_negotiation.cc:407] Connection header received
0619 10:18:43.574569 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:18:43.574570 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:18:43.574590 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:18:43.574684 (+ 94us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 10:18:43.574840 (+ 156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:18:43.575491 (+ 651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:18:43.576606 (+ 1115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:18:43.576927 (+ 321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:18:43.576983 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:18:43.577169 (+ 186us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 10:18:43.577301 (+ 132us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 10:18:43.577372 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0619 10:18:43.577590 (+ 218us) server_negotiation.cc:299] Negotiation successful
0619 10:18:43.577607 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260619 10:18:43.579486 12442 raft_consensus.cc:1261] T 7b8505d957f0476aaca23896ec9c99bf P a6d4bbbd26b24bfab203c0fdbce9b4ec [term 1 FOLLOWER]: Refusing update from remote peer b461caa3dc3741278f936964a3edfe88: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 10:18:43.579615 12305 raft_consensus.cc:1261] T 7b8505d957f0476aaca23896ec9c99bf P d7c17b58dde54337ba12510a19f60697 [term 1 FOLLOWER]: Refusing update from remote peer b461caa3dc3741278f936964a3edfe88: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 10:18:43.579875 12520 consensus_queue.cc:1035] T 7b8505d957f0476aaca23896ec9c99bf P b461caa3dc3741278f936964a3edfe88 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a6d4bbbd26b24bfab203c0fdbce9b4ec" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 42949 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 10:18:43.580024 12515 consensus_queue.cc:1035] T 7b8505d957f0476aaca23896ec9c99bf P b461caa3dc3741278f936964a3edfe88 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d7c17b58dde54337ba12510a19f60697" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 38849 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 10:18:43.583523 12525 mvcc.cc:205] Tried to move back new op lower bound from 7298516269379080192 to 7298516269308239872. Current Snapshot: MvccSnapshot[applied={T|T < 7298516269379080192}]
I20260619 10:18:43.584149 12524 mvcc.cc:205] Tried to move back new op lower bound from 7298516269379080192 to 7298516269308239872. Current Snapshot: MvccSnapshot[applied={T|T < 7298516269379080192}]
I20260619 10:18:43.589020 12004 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56674:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 10:18:43.589107 12004 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56674:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 10:18:43.589888 12004 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a2115a1d164b42e49e7d16df95ca9294: Sending DeleteTablet for 3 replicas of tablet 7b8505d957f0476aaca23896ec9c99bf
I20260619 10:18:43.590404 12417 tablet_service.cc:1514] Processing DeleteTablet for tablet 7b8505d957f0476aaca23896ec9c99bf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 10:18:43 UTC) from {username='kudu', principal='kudu/127.0.206.126@KRBTEST.COM'} at 127.0.0.1:39116
I20260619 10:18:43.591006 12288 tablet_service.cc:1514] Processing DeleteTablet for tablet 7b8505d957f0476aaca23896ec9c99bf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 10:18:43 UTC) from {username='kudu', principal='kudu/127.0.206.126@KRBTEST.COM'} at 127.0.0.1:33884
I20260619 10:18:43.591850 825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTU9OP6/build/release/bin/kudu with pid 12085
I20260619 10:18:43.593422 12533 tablet_replica.cc:330] T 7b8505d957f0476aaca23896ec9c99bf P d7c17b58dde54337ba12510a19f60697: stopping tablet replica
I20260619 10:18:43.593542 12533 raft_consensus.cc:2229] T 7b8505d957f0476aaca23896ec9c99bf P d7c17b58dde54337ba12510a19f60697 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 10:18:43.593659 12533 raft_consensus.cc:2258] T 7b8505d957f0476aaca23896ec9c99bf P d7c17b58dde54337ba12510a19f60697 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 10:18:43.593832 12532 tablet_replica.cc:330] T 7b8505d957f0476aaca23896ec9c99bf P a6d4bbbd26b24bfab203c0fdbce9b4ec: stopping tablet replica
I20260619 10:18:43.593917 12532 raft_consensus.cc:2229] T 7b8505d957f0476aaca23896ec9c99bf P a6d4bbbd26b24bfab203c0fdbce9b4ec [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 10:18:43.594003 12532 raft_consensus.cc:2258] T 7b8505d957f0476aaca23896ec9c99bf P a6d4bbbd26b24bfab203c0fdbce9b4ec [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 10:18:43.594425 12533 ts_tablet_manager.cc:1905] T 7b8505d957f0476aaca23896ec9c99bf P d7c17b58dde54337ba12510a19f60697: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 10:18:43.594455 12532 ts_tablet_manager.cc:1905] T 7b8505d957f0476aaca23896ec9c99bf P a6d4bbbd26b24bfab203c0fdbce9b4ec: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 10:18:43.596207 12533 ts_tablet_manager.cc:1918] T 7b8505d957f0476aaca23896ec9c99bf P d7c17b58dde54337ba12510a19f60697: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 10:18:43.596374 12533 log.cc:1196] T 7b8505d957f0476aaca23896ec9c99bf P d7c17b58dde54337ba12510a19f60697: Deleting WAL directory at /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781864224577058-825-0/minicluster-data/ts-1/wal/wals/7b8505d957f0476aaca23896ec9c99bf
I20260619 10:18:43.596577 12532 ts_tablet_manager.cc:1918] T 7b8505d957f0476aaca23896ec9c99bf P a6d4bbbd26b24bfab203c0fdbce9b4ec: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 10:18:43.596701 12533 ts_tablet_manager.cc:1939] T 7b8505d957f0476aaca23896ec9c99bf P d7c17b58dde54337ba12510a19f60697: Deleting consensus metadata
I20260619 10:18:43.596717 12532 log.cc:1196] T 7b8505d957f0476aaca23896ec9c99bf P a6d4bbbd26b24bfab203c0fdbce9b4ec: Deleting WAL directory at /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781864224577058-825-0/minicluster-data/ts-2/wal/wals/7b8505d957f0476aaca23896ec9c99bf
I20260619 10:18:43.597033 12532 ts_tablet_manager.cc:1939] T 7b8505d957f0476aaca23896ec9c99bf P a6d4bbbd26b24bfab203c0fdbce9b4ec: Deleting consensus metadata
I20260619 10:18:43.597216 11990 catalog_manager.cc:4841] TS d7c17b58dde54337ba12510a19f60697 (127.0.206.66:38849): tablet 7b8505d957f0476aaca23896ec9c99bf (table test-table [id=91ff76a6ce4d4b98aadc625b72846a2d]) successfully deleted
I20260619 10:18:43.597467 11989 catalog_manager.cc:4841] TS a6d4bbbd26b24bfab203c0fdbce9b4ec (127.0.206.67:42949): tablet 7b8505d957f0476aaca23896ec9c99bf (table test-table [id=91ff76a6ce4d4b98aadc625b72846a2d]) successfully deleted
W20260619 10:18:43.602588 11989 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.0.206.65:34117 (error 108)
I20260619 10:18:43.603411 12501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:18:43.603278 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.65:34117 (local address 127.0.0.1:47796)
0619 10:18:43.603336 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0619 10:18:43.603376 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.65:34117: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260619 10:18:43.603439 825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTU9OP6/build/release/bin/kudu with pid 12224
W20260619 10:18:43.603518 11989 catalog_manager.cc:4584] TS b461caa3dc3741278f936964a3edfe88 (127.0.206.65:34117): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 7b8505d957f0476aaca23896ec9c99bf: Network error: Client connection negotiation failed: client connection to 127.0.206.65:34117: connect: Connection refused (error 111)
I20260619 10:18:43.609086 825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTU9OP6/build/release/bin/kudu with pid 12359
I20260619 10:18:43.614840 825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTU9OP6/build/release/bin/kudu with pid 11972
I20260619 10:18:43.621127 825 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260619 10:18:43.955818 825 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260619 10:18:43.955914 825 mini_ranger.cc:66] Stopping Ranger...
I20260619 10:18:44.302812 825 mini_ranger.cc:68] Stopped Ranger
2026-06-19 10:18:44.303 UTC [11558] LOG: received smart shutdown request
2026-06-19 10:18:44.306 UTC [11558] LOG: background worker "logical replication launcher" (PID 11567) exited with exit code 1
2026-06-19 10:18:44.306 UTC [11562] LOG: shutting down
2026-06-19 10:18:44.447 UTC [11558] LOG: database system is shut down
2026-06-19T10:18:44Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (44860 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781864224577058-825-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 10:18:44 dist-test-slave-6pt2 krb5kdc[12544](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 10:18:44 dist-test-slave-6pt2 krb5kdc[12544](info): set up 2 sockets
Jun 19 10:18:44 dist-test-slave-6pt2 krb5kdc[12544](info): commencing operation
krb5kdc: starting...
W20260619 10:18:45.602684 825 mini_kdc.cc:121] Time spent starting KDC: real 1.023s user 0.005s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781864224577058-825-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781864224577058-825-0/krb5kdc/test-user.keytab.
Jun 19 10:18:45 dist-test-slave-6pt2 krb5kdc[12544](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864325, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T10:18:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T10:18:45Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.0.206.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.0.206.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.0.206.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781864224577058-825-0/krb5kdc/rangeradmin_127.0.206.84@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.0.206.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781864224577058-825-0/krb5kdc/rangeradmin_127.0.206.84@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.0.206.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.0.206.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.0.206.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781864224577058-825-0/krb5kdc/rangerlookup_127.0.206.84@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.0.206.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781864224577058-825-0/krb5kdc/rangerlookup_127.0.206.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.0.206.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.0.206.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.0.206.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781864224577058-825-0/krb5kdc/HTTP_127.0.206.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.0.206.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781864224577058-825-0/krb5kdc/HTTP_127.0.206.84@KRBTEST.COM.keytab.
I20260619 10:18:45.754565 825 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781864224577058-825-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskTU9OP6/build/release/bin/postgres/pg_ctl -D /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781864224577058-825-0/minicluster-data/postgres -l logfile start
2026-06-19 10:18:48.049 UTC [12574] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-06-19 10:18:48.049 UTC [12574] LOG: listening on IPv4 address "127.0.206.84", port 57005
2026-06-19 10:18:48.051 UTC [12574] LOG: listening on Unix socket "/tmp/.s.PGSQL.57005"
2026-06-19 10:18:48.071 UTC [12577] LOG: database system was shut down at 2026-06-19 10:18:46 UTC
2026-06-19 10:18:48.074 UTC [12574] LOG: database system is ready to accept connections
I20260619 10:18:49.030485 825 mini_postgres.cc:96] Postgres bound to 57005
2026-06-19 10:18:49.036 UTC [12585] FATAL: database "slave" does not exist
127.0.206.84:57005 - accepting connections
I20260619 10:18:49.036793 825 mini_ranger.cc:161] Starting Ranger...
I20260619 10:18:49.050858 825 mini_ranger.cc:84] Created miniranger Postgres user
I20260619 10:18:49.457871 825 mini_ranger.cc:87] Created ranger Postgres database
I20260619 10:18:49.457937 825 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781864224577058-825-0/minicluster-data/ranger-admin
2026-06-19 10:18:49,741 [I] DB FLAVOR :POSTGRES
2026-06-19 10:18:49,741 [I] --------- Verifying Ranger DB connection ---------
2026-06-19 10:18:49,741 [I] Checking connection..
2026-06-19 10:18:49,741 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskTU9OP6/build/release/bin/postgresql.jar:/tmp/dist-test-taskTU9OP6/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.206.84:57005/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-06-19 10:18:49,969 [I] Checking connection passed.
2026-06-19 10:18:49,970 [I] --------- Verifying version history table ---------
2026-06-19 10:18:49,970 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskTU9OP6/build/release/bin/postgresql.jar:/tmp/dist-test-taskTU9OP6/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.206.84:57005/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-06-19 10:18:50,197 [I] Table x_db_version_h does not exist in database ranger
2026-06-19 10:18:50,197 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskTU9OP6/build/release/bin/postgresql.jar:/tmp/dist-test-taskTU9OP6/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.206.84:57005/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-06-19 10:18:50,449 [I] Table x_db_version_h does not exist in database ranger
2026-06-19 10:18:50,449 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-06-19 10:18:50,449 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskTU9OP6/build/release/bin/postgresql.jar:/tmp/dist-test-taskTU9OP6/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.206.84:57005/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskTU9OP6/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-06-19 10:18:50.687 UTC [12649] WARNING: there is no transaction in progress
2026-06-19 10:18:50,693 [I] create_dbversion_catalog.sql file imported successfully
2026-06-19 10:18:50,693 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskTU9OP6/build/release/bin/postgresql.jar:/tmp/dist-test-taskTU9OP6/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.206.84:57005/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-06-19 10:18:50,927 [I] Table x_db_version_h already exists in database 'ranger'
2026-06-19 10:18:50,927 [I] --------- Importing Ranger Core DB Schema ---------
2026-06-19 10:18:50,927 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskTU9OP6/build/release/bin/postgresql.jar:/tmp/dist-test-taskTU9OP6/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.206.84:57005/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-06-19 10:18:51,183 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskTU9OP6/build/release/bin/postgresql.jar:/tmp/dist-test-taskTU9OP6/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.206.84:57005/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-06-19 10:18:51,436 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskTU9OP6/build/release/bin/postgresql.jar:/tmp/dist-test-taskTU9OP6/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.206.84:57005/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-6pt2.c.gcp-upstream.internal','N') ;"
2026-06-19 10:18:51,655 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskTU9OP6/build/release/bin/postgresql.jar:/tmp/dist-test-taskTU9OP6/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.206.84:57005/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-06-19 10:18:51,879 [I] Table x_portal_user does not exist in database ranger
2026-06-19 10:18:51,879 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskTU9OP6/build/release/bin/postgresql.jar:/tmp/dist-test-taskTU9OP6/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.206.84:57005/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-06-19 10:18:52,119 [I] Table x_policy_ref_group does not exist in database ranger
2026-06-19 10:18:52,119 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskTU9OP6/build/release/bin/postgresql.jar:/tmp/dist-test-taskTU9OP6/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.206.84:57005/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-06-19 10:18:52,373 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskTU9OP6/build/release/bin/postgresql.jar:/tmp/dist-test-taskTU9OP6/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.206.84:57005/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-06-19 10:18:52,623 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-06-19 10:18:52,623 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskTU9OP6/build/release/bin/postgresql.jar:/tmp/dist-test-taskTU9OP6/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.206.84:57005/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskTU9OP6/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-06-19 10:18:53.521 UTC [12757] WARNING: there is no transaction in progress
2026-06-19 10:18:53.536 UTC [12757] WARNING: there is no transaction in progress
2026-06-19 10:18:53.551 UTC [12757] WARNING: there is no transaction in progress
2026-06-19 10:18:53.565 UTC [12757] WARNING: there is no transaction in progress
2026-06-19 10:18:53.582 UTC [12757] WARNING: there is no transaction in progress
2026-06-19 10:18:53.598 UTC [12757] WARNING: there is no transaction in progress
2026-06-19 10:18:53.648 UTC [12757] WARNING: there is no transaction in progress
2026-06-19 10:18:53.654 UTC [12757] WARNING: there is no transaction in progress
2026-06-19 10:18:53.668 UTC [12757] WARNING: there is no transaction in progress
2026-06-19 10:18:53.677 UTC [12757] WARNING: there is no transaction in progress
2026-06-19 10:18:53.687 UTC [12757] WARNING: there is no transaction in progress
2026-06-19 10:18:53.699 UTC [12757] WARNING: there is no transaction in progress
2026-06-19 10:18:53.709 UTC [12757] WARNING: there is no transaction in progress
2026-06-19 10:18:53.715 UTC [12757] WARNING: there is no transaction in progress
2026-06-19 10:18:53.721 UTC [12757] WARNING: there is no transaction in progress
2026-06-19 10:18:53.727 UTC [12757] WARNING: there is no transaction in progress
2026-06-19 10:18:54,300 [I] ranger_core_db_postgres.sql file imported successfully
2026-06-19 10:18:54,300 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskTU9OP6/build/release/bin/postgresql.jar:/tmp/dist-test-taskTU9OP6/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.206.84:57005/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-06-19 10:18:54,550 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-06-19 10:18:54,551 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskTU9OP6/build/release/bin/postgresql.jar:/tmp/dist-test-taskTU9OP6/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.206.84:57005/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-06-19 10:18:54,784 [I] Table x_portal_user already exists in database 'ranger'
2026-06-19 10:18:54,785 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskTU9OP6/build/release/bin/postgresql.jar:/tmp/dist-test-taskTU9OP6/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.206.84:57005/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-06-19 10:18:55,038 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-06-19 10:18:55,039 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskTU9OP6/build/release/bin/postgresql.jar:/tmp/dist-test-taskTU9OP6/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.206.84:57005/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-06-19 10:18:55,275 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskTU9OP6/build/release/bin/postgresql.jar:/tmp/dist-test-taskTU9OP6/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.206.84:57005/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-06-19 10:18:55,530 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskTU9OP6/build/release/bin/postgresql.jar:/tmp/dist-test-taskTU9OP6/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.206.84:57005/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-6pt2.c.gcp-upstream.internal';"
2026-06-19 10:18:55,773 [I] CORE_DB_SCHEMA import status has been updated
2026-06-19 10:18:55,774 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskTU9OP6/build/release/bin/postgresql.jar:/tmp/dist-test-taskTU9OP6/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.206.84:57005/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-06-19 10:18:56,008 [I] DB_PATCHES have already been applied
I20260619 10:18:56.014377 825 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781864224577058-825-0/minicluster-data/ranger-admin:/tmp/dist-test-taskTU9OP6/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskTU9OP6/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskTU9OP6/build/release/bin/postgresql.jar:/tmp/dist-test-taskTU9OP6/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260619 10:18:56.014467 825 mini_ranger.cc:193] Using host: 127.0.206.84
I20260619 10:18:56.017424 825 mini_ranger.cc:227] Ranger admin URL: http://127.0.206.84:60039
Jun 19, 2026 10:18:56 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Jun 19, 2026 10:18:56 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Jun 19, 2026 10:18:56 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781864224577058-825-0/minicluster-data/ranger-admin/ews/webapp
Jun 19, 2026 10:18:56 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781864224577058-825-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Jun 19, 2026 10:18:56 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781864224577058-825-0/minicluster-data/ranger-admin/ews/webapp] .....
Jun 19, 2026 10:18:56 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781864224577058-825-0/minicluster-data/ranger-admin/ews/webapp].
Jun 19, 2026 10:18:56 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260619 10:18:57.140965 825 mini_ranger.cc:160] Time spent starting Ranger: real 8.104s user 0.000s sys 0.008s
Jun 19, 2026 10:19:15 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Jun 19, 2026 10:19:15 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.UserREST
Jun 19, 2026 10:19:15 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Jun 19, 2026 10:19:15 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Jun 19, 2026 10:19:15 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Jun 19, 2026 10:19:15 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Jun 19, 2026 10:19:15 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Jun 19, 2026 10:19:15 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Jun 19, 2026 10:19:15 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Jun 19, 2026 10:19:15 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Jun 19, 2026 10:19:15 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Jun 19, 2026 10:19:15 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Jun 19, 2026 10:19:15 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Jun 19, 2026 10:19:15 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Jun 19, 2026 10:19:15 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Jun 19, 2026 10:19:15 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Jun 19, 2026 10:19:15 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Jun 19, 2026 10:19:16 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260619 10:19:16.954689 825 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.0.206.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.0.206.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.0.206.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781864224577058-825-0/krb5kdc/rangerkms_127.0.206.84@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.0.206.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781864224577058-825-0/krb5kdc/rangerkms_127.0.206.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.0.206.84@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.0.206.84@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.0.206.84@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781864224577058-825-0/krb5kdc/HTTP_127.0.206.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.0.206.84@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781864224577058-825-0/krb5kdc/HTTP_127.0.206.84@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Jun 19 10:19:17 dist-test-slave-6pt2 krb5kdc[12544](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864357, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260619 10:19:17.026894 825 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260619 10:19:17.035648 825 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260619 10:19:17.797778 825 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260619 10:19:17.797859 825 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781864224577058-825-0/minicluster-data/ranger-kms
I20260619 10:19:17.797883 825 mini_ranger_kms.cc:227] Using postgres at 127.0.206.84:57005
2026-06-19 10:19:17,956 [I] DB FLAVOR :POSTGRES
2026-06-19 10:19:17,957 [I] --------- Verifying Ranger DB connection ---------
2026-06-19 10:19:17,957 [I] Checking connection
2026-06-19 10:19:17,957 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskTU9OP6/build/release/bin/postgresql.jar:/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781864224577058-825-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.206.84:57005/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-06-19 10:19:18,195 [I] connection success
2026-06-19 10:19:18,195 [I] --------- Verifying Ranger DB tables ---------
2026-06-19 10:19:18,195 [I] Verifying table ranger_masterkey in database rangerkms
2026-06-19 10:19:18,196 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskTU9OP6/build/release/bin/postgresql.jar:/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781864224577058-825-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.206.84:57005/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-06-19 10:19:18,436 [I] Table ranger_masterkey does not exist in database rangerkms
2026-06-19 10:19:18,436 [I] --------- Importing Ranger Core DB Schema ---------
2026-06-19 10:19:18,436 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-06-19 10:19:18,436 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskTU9OP6/build/release/bin/postgresql.jar:/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781864224577058-825-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.0.206.84:57005/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781864224577058-825-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-06-19 10:19:18,682 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260619 10:19:19.145756 825 mini_ranger_kms.cc:310] Created kms service
I20260619 10:19:19.333076 825 mini_ranger_kms.cc:326] Created kudu user
I20260619 10:19:19.376325 825 mini_ranger_kms.cc:343] Created rangerkms user
Jun 19 10:19:19 dist-test-slave-6pt2 krb5kdc[12544](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864357, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.0.206.84@KRBTEST.COM
I20260619 10:19:19.758934 825 mini_ranger_kms.cc:384] Added ranger policy
I20260619 10:19:19.759045 825 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781864224577058-825-0/minicluster-data/ranger-kms:/tmp/dist-test-taskTU9OP6/build/release/bin/postgresql.jar:/tmp/dist-test-taskTU9OP6/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskTU9OP6/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260619 10:19:19.759063 825 mini_ranger_kms.cc:240] Using host: 127.0.206.84
I20260619 10:19:19.763710 825 mini_ranger_kms.cc:276] Ranger KMS PID: 12937
I20260619 10:19:19.763794 825 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.0.206.84:59075
Jun 19, 2026 10:19:20 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Jun 19, 2026 10:19:20 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Jun 19, 2026 10:19:20 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781864224577058-825-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Jun 19, 2026 10:19:20 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781864224577058-825-0/minicluster-data/ranger-kms/ews/webapp] .....
Jun 19, 2026 10:19:21 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781864224577058-825-0/minicluster-data/ranger-kms/ews/webapp].
Jun 19, 2026 10:19:21 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
I20260619 10:19:21.929181 825 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 4.902s user 0.007s sys 0.134s
I20260619 10:19:21.929291 825 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260619 10:19:21.929347 825 mini_ranger_kms.cc:401] 127.0.206.84:59075/kms/v1/keys
Jun 19 10:19:21 dist-test-slave-6pt2 krb5kdc[12544](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1781864361, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.0.206.84@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Jun 19 10:19:22 dist-test-slave-6pt2 krb5kdc[12544](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864361, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.0.206.84@KRBTEST.COM for HTTP/127.0.206.84@KRBTEST.COM
Jun 19 10:19:22 dist-test-slave-6pt2 krb5kdc[12544](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864361, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.0.206.84@KRBTEST.COM for HTTP/127.0.206.84@KRBTEST.COM
Jun 19 10:19:24 dist-test-slave-6pt2 krb5kdc[12544](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864364, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.0.206.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.206.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.206.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781864224577058-825-0/krb5kdc/kudu_127.0.206.126.keytab.
Entry for principal kudu/127.0.206.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781864224577058-825-0/krb5kdc/kudu_127.0.206.126.keytab.
I20260619 10:19:24.990933 825 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTU9OP6/build/release/bin/kudu
/tmp/dist-test-taskTU9OP6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781864224577058-825-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781864224577058-825-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781864224577058-825-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781864224577058-825-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.206.126:43433
--webserver_interface=127.0.206.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781864224577058-825-0/krb5kdc/kudu_127.0.206.126.keytab
--principal=kudu/127.0.206.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.206.84:39201
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.206.126:43433
--ranger_config_path=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781864224577058-825-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.0.206.84:59075/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781864224577058-825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781864224577058-825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781864224577058-825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 10:19:25.102047 12988 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 10:19:25.102274 12988 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 10:19:25.102310 12988 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 10:19:25.103560 12988 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 10:19:25.103628 12988 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 10:19:25.103672 12988 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 10:19:25.103721 12988 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 10:19:25.103751 12988 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 10:19:25.105283 12988 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.206.84:39201
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781864224577058-825-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781864224577058-825-0/minicluster-data/master-0/wal
--ranger_kms_url=127.0.206.84:59075/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.0.206.126:43433
--ranger_config_path=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781864224577058-825-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781864224577058-825-0/krb5kdc/kudu_127.0.206.126.keytab
--principal=kudu/127.0.206.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.206.126:43433
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781864224577058-825-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.206.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12988
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781864224577058-825-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 09:32:44 UTC on ccycloud.quasar-gbtldp.root.comops.site
build id 4455440
I20260619 10:19:25.105551 12988 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 10:19:25.105772 12988 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 10:19:25.111770 12996 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 10:19:25.111778 12993 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 10:19:25.111891 12988 server_base.cc:1034] running on GCE node
W20260619 10:19:25.111783 12994 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 10:19:25.112308 12988 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 10:19:25.112596 12988 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 10:19:25.113760 12988 hybrid_clock.cc:648] HybridClock initialized: now 1781864365113739 us; error 38 us; skew 500 ppm
Jun 19 10:19:25 dist-test-slave-6pt2 krb5kdc[12544](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864365, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 10:19:25.119298 12988 init.cc:377] Logged in from keytab as kudu/127.0.206.126@KRBTEST.COM (short username kudu)
I20260619 10:19:25.119807 12988 webserver.cc:460] Webserver started at http://127.0.206.126:39425/ using document root <none> and password file <none>
I20260619 10:19:25.120033 12988 fs_manager.cc:359] Metadata directory not provided
I20260619 10:19:25.120095 12988 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 10:19:25.120227 12988 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Jun 19 10:19:25 dist-test-slave-6pt2 krb5kdc[12544](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864365, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.126@KRBTEST.COM for HTTP/127.0.206.84@KRBTEST.COM
I20260619 10:19:25.174376 12988 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781864224577058-825-0/minicluster-data/master-0/data/instance:
uuid: "73991fccce974540bd47e4cfdc79aea7"
format_stamp: "Formatted at 2026-06-19 10:19:25 on dist-test-slave-6pt2"
server_key: "eb6f28be637b1dea25f38f297118f7f0"
server_key_iv: "8c31fcb84bac5de0c0679961690207df"
server_key_version: "kuduclusterkey@0"
I20260619 10:19:25.174799 12988 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781864224577058-825-0/minicluster-data/master-0/wal/instance:
uuid: "73991fccce974540bd47e4cfdc79aea7"
format_stamp: "Formatted at 2026-06-19 10:19:25 on dist-test-slave-6pt2"
server_key: "eb6f28be637b1dea25f38f297118f7f0"
server_key_iv: "8c31fcb84bac5de0c0679961690207df"
server_key_version: "kuduclusterkey@0"
I20260619 10:19:25.176318 12988 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260619 10:19:25.191151 13003 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 10:19:25.191326 12988 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260619 10:19:25.191411 12988 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781864224577058-825-0/minicluster-data/master-0/data,/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781864224577058-825-0/minicluster-data/master-0/wal
uuid: "73991fccce974540bd47e4cfdc79aea7"
format_stamp: "Formatted at 2026-06-19 10:19:25 on dist-test-slave-6pt2"
server_key: "eb6f28be637b1dea25f38f297118f7f0"
server_key_iv: "8c31fcb84bac5de0c0679961690207df"
server_key_version: "kuduclusterkey@0"
I20260619 10:19:25.191512 12988 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781864224577058-825-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781864224577058-825-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781864224577058-825-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 10:19:25.219553 12988 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 10:19:25.221558 12988 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 10:19:25.221706 12988 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 10:19:25.226552 12988 rpc_server.cc:307] RPC server started. Bound to: 127.0.206.126:43433
I20260619 10:19:25.226649 13055 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.206.126:43433 every 8 connection(s)
I20260619 10:19:25.227013 12988 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781864224577058-825-0/minicluster-data/master-0/data/info.pb
I20260619 10:19:25.227319 825 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTU9OP6/build/release/bin/kudu as pid 12988
I20260619 10:19:25.227432 825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781864224577058-825-0/minicluster-data/master-0/wal/instance
I20260619 10:19:25.227626 825 external_mini_cluster.cc:1428] Setting key c1450294495137c00fd9a5035b32ddda
I20260619 10:19:25.227813 13056 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 10:19:25.230075 13056 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 73991fccce974540bd47e4cfdc79aea7: Bootstrap starting.
I20260619 10:19:25.230787 13056 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 73991fccce974540bd47e4cfdc79aea7: Neither blocks nor log segments found. Creating new log.
I20260619 10:19:25.231113 13056 log.cc:824] T 00000000000000000000000000000000 P 73991fccce974540bd47e4cfdc79aea7: Log is configured to *not* fsync() on all Append() calls
I20260619 10:19:25.231978 13056 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 73991fccce974540bd47e4cfdc79aea7: No bootstrap required, opened a new log
I20260619 10:19:25.234304 13056 raft_consensus.cc:348] T 00000000000000000000000000000000 P 73991fccce974540bd47e4cfdc79aea7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "73991fccce974540bd47e4cfdc79aea7" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 43433 } }
I20260619 10:19:25.234439 13056 raft_consensus.cc:374] T 00000000000000000000000000000000 P 73991fccce974540bd47e4cfdc79aea7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 10:19:25.234462 13056 raft_consensus.cc:729] T 00000000000000000000000000000000 P 73991fccce974540bd47e4cfdc79aea7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 73991fccce974540bd47e4cfdc79aea7, State: Initialized, Role: FOLLOWER
I20260619 10:19:25.234632 13056 consensus_queue.cc:260] T 00000000000000000000000000000000 P 73991fccce974540bd47e4cfdc79aea7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "73991fccce974540bd47e4cfdc79aea7" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 43433 } }
I20260619 10:19:25.234719 13056 raft_consensus.cc:388] T 00000000000000000000000000000000 P 73991fccce974540bd47e4cfdc79aea7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 10:19:25.234751 13056 raft_consensus.cc:482] T 00000000000000000000000000000000 P 73991fccce974540bd47e4cfdc79aea7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 10:19:25.234802 13056 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 73991fccce974540bd47e4cfdc79aea7 [term 0 FOLLOWER]: Advancing to term 1
I20260619 10:19:25.235500 13056 raft_consensus.cc:504] T 00000000000000000000000000000000 P 73991fccce974540bd47e4cfdc79aea7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "73991fccce974540bd47e4cfdc79aea7" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 43433 } }
I20260619 10:19:25.235618 13056 leader_election.cc:302] T 00000000000000000000000000000000 P 73991fccce974540bd47e4cfdc79aea7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 73991fccce974540bd47e4cfdc79aea7; no voters:
I20260619 10:19:25.235791 13056 leader_election.cc:288] T 00000000000000000000000000000000 P 73991fccce974540bd47e4cfdc79aea7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 10:19:25.235852 13061 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 73991fccce974540bd47e4cfdc79aea7 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 10:19:25.236001 13061 raft_consensus.cc:686] T 00000000000000000000000000000000 P 73991fccce974540bd47e4cfdc79aea7 [term 1 LEADER]: Becoming Leader. State: Replica: 73991fccce974540bd47e4cfdc79aea7, State: Running, Role: LEADER
I20260619 10:19:25.236086 13056 sys_catalog.cc:564] T 00000000000000000000000000000000 P 73991fccce974540bd47e4cfdc79aea7 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 10:19:25.236090 13061 consensus_queue.cc:237] T 00000000000000000000000000000000 P 73991fccce974540bd47e4cfdc79aea7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "73991fccce974540bd47e4cfdc79aea7" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 43433 } }
I20260619 10:19:25.236415 13062 sys_catalog.cc:455] T 00000000000000000000000000000000 P 73991fccce974540bd47e4cfdc79aea7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "73991fccce974540bd47e4cfdc79aea7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "73991fccce974540bd47e4cfdc79aea7" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 43433 } } }
I20260619 10:19:25.236495 13062 sys_catalog.cc:458] T 00000000000000000000000000000000 P 73991fccce974540bd47e4cfdc79aea7 [sys.catalog]: This master's current role is: LEADER
I20260619 10:19:25.236743 13063 sys_catalog.cc:455] T 00000000000000000000000000000000 P 73991fccce974540bd47e4cfdc79aea7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 73991fccce974540bd47e4cfdc79aea7. Latest consensus state: current_term: 1 leader_uuid: "73991fccce974540bd47e4cfdc79aea7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "73991fccce974540bd47e4cfdc79aea7" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 43433 } } }
I20260619 10:19:25.236856 13063 sys_catalog.cc:458] T 00000000000000000000000000000000 P 73991fccce974540bd47e4cfdc79aea7 [sys.catalog]: This master's current role is: LEADER
Jun 19 10:19:25 dist-test-slave-6pt2 krb5kdc[12544](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864364, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.206.126@KRBTEST.COM
I20260619 10:19:25.237066 13067 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 10:19:25.237582 13067 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 10:19:25.237850 13056 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781864224577058-825-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260619 10:19:25.243530 13067 catalog_manager.cc:1347] Generated new cluster ID: 52c9af318b874300ba817d9f016418d4
I20260619 10:19:25.243602 13067 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 10:19:25.245262 13059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:19:25.228123 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51298 (local address 127.0.206.126:43433)
0619 10:19:25.228322 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0619 10:19:25.228331 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 10:19:25.230385 (+ 2054us) server_negotiation.cc:407] Connection header received
0619 10:19:25.231032 (+ 647us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:19:25.231042 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:19:25.231106 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:19:25.231282 (+ 176us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:19:25.232111 (+ 829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:19:25.232969 (+ 858us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:19:25.234119 (+ 1150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:19:25.234603 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:19:25.234747 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:19:25.238175 (+ 3428us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 10:19:25.238179 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 10:19:25.238179 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 10:19:25.238189 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0619 10:19:25.243103 (+ 4914us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:19:25.243526 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:19:25.243529 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:19:25.243530 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:19:25.243763 (+ 233us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:19:25.243987 (+ 224us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:19:25.243988 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:19:25.243989 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:19:25.244275 (+ 286us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 10:19:25.244420 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0619 10:19:25.244822 (+ 402us) server_negotiation.cc:299] Negotiation successful
0619 10:19:25.244987 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":135,"thread_start_us":77,"threads_started":1}
I20260619 10:19:25.249178 13067 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 10:19:25.249673 13067 catalog_manager.cc:1504] Loading token signing keys...
I20260619 10:19:25.257333 13067 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 73991fccce974540bd47e4cfdc79aea7: Generated new TSK 0
I20260619 10:19:25.257495 13067 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.0.206.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.206.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.206.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781864224577058-825-0/krb5kdc/kudu_127.0.206.65.keytab.
Entry for principal kudu/127.0.206.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781864224577058-825-0/krb5kdc/kudu_127.0.206.65.keytab.
I20260619 10:19:25.970275 825 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTU9OP6/build/release/bin/kudu
/tmp/dist-test-taskTU9OP6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781864224577058-825-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781864224577058-825-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781864224577058-825-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781864224577058-825-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.0.206.84:59075/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.206.65:0
--local_ip_for_outbound_sockets=127.0.206.65
--webserver_interface=127.0.206.65
--webserver_port=0
--tserver_master_addrs=127.0.206.126:43433
--keytab_file=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781864224577058-825-0/krb5kdc/kudu_127.0.206.65.keytab
--principal=kudu/127.0.206.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.206.84:39201
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781864224577058-825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781864224577058-825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781864224577058-825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 10:19:26.054294 13101 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 10:19:26.054551 13101 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 10:19:26.054616 13101 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 10:19:26.056324 13101 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 10:19:26.056418 13101 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 10:19:26.056474 13101 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.206.65
I20260619 10:19:26.058696 13101 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.206.84:39201
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781864224577058-825-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781864224577058-825-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.0.206.84:59075/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781864224577058-825-0/krb5kdc/kudu_127.0.206.65.keytab
--principal=kudu/127.0.206.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.206.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781864224577058-825-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.206.65
--webserver_port=0
--tserver_master_addrs=127.0.206.126:43433
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13101
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.206.65
--log_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781864224577058-825-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 09:32:44 UTC on ccycloud.quasar-gbtldp.root.comops.site
build id 4455440
I20260619 10:19:26.058924 13101 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 10:19:26.059190 13101 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 10:19:26.065189 13107 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 10:19:26.065227 13109 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 10:19:26.065299 13106 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 10:19:26.065625 13101 server_base.cc:1034] running on GCE node
I20260619 10:19:26.065836 13101 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 10:19:26.066098 13101 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 10:19:26.067252 13101 hybrid_clock.cc:648] HybridClock initialized: now 1781864366067239 us; error 29 us; skew 500 ppm
Jun 19 10:19:26 dist-test-slave-6pt2 krb5kdc[12544](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864366, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 10:19:26.069562 13101 init.cc:377] Logged in from keytab as kudu/127.0.206.65@KRBTEST.COM (short username kudu)
I20260619 10:19:26.070009 13101 webserver.cc:460] Webserver started at http://127.0.206.65:34033/ using document root <none> and password file <none>
I20260619 10:19:26.070259 13101 fs_manager.cc:359] Metadata directory not provided
I20260619 10:19:26.070299 13101 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 10:19:26.070406 13101 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Jun 19 10:19:26 dist-test-slave-6pt2 krb5kdc[12544](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864366, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.65@KRBTEST.COM for HTTP/127.0.206.84@KRBTEST.COM
I20260619 10:19:26.083868 13101 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781864224577058-825-0/minicluster-data/ts-0/data/instance:
uuid: "c31c3e7e16184ade9825d9bd931f460f"
format_stamp: "Formatted at 2026-06-19 10:19:26 on dist-test-slave-6pt2"
server_key: "066650d496f4735e5ea78ec6225919da"
server_key_iv: "0d3cb177a7961e0632b747e2bdd25359"
server_key_version: "kuduclusterkey@0"
I20260619 10:19:26.084313 13101 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781864224577058-825-0/minicluster-data/ts-0/wal/instance:
uuid: "c31c3e7e16184ade9825d9bd931f460f"
format_stamp: "Formatted at 2026-06-19 10:19:26 on dist-test-slave-6pt2"
server_key: "066650d496f4735e5ea78ec6225919da"
server_key_iv: "0d3cb177a7961e0632b747e2bdd25359"
server_key_version: "kuduclusterkey@0"
I20260619 10:19:26.087015 13101 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.001s sys 0.000s
I20260619 10:19:26.102735 13116 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 10:19:26.102900 13101 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.001s
I20260619 10:19:26.102988 13101 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781864224577058-825-0/minicluster-data/ts-0/data,/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781864224577058-825-0/minicluster-data/ts-0/wal
uuid: "c31c3e7e16184ade9825d9bd931f460f"
format_stamp: "Formatted at 2026-06-19 10:19:26 on dist-test-slave-6pt2"
server_key: "066650d496f4735e5ea78ec6225919da"
server_key_iv: "0d3cb177a7961e0632b747e2bdd25359"
server_key_version: "kuduclusterkey@0"
I20260619 10:19:26.103085 13101 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781864224577058-825-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781864224577058-825-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781864224577058-825-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 10:19:26.121565 13101 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 10:19:26.124110 13101 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 10:19:26.124274 13101 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 10:19:26.124503 13101 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 10:19:26.124861 13101 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 10:19:26.124914 13101 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 10:19:26.124953 13101 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 10:19:26.124975 13101 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 10:19:26.131711 13101 rpc_server.cc:307] RPC server started. Bound to: 127.0.206.65:42679
I20260619 10:19:26.132143 13101 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781864224577058-825-0/minicluster-data/ts-0/data/info.pb
I20260619 10:19:26.132694 13231 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.206.65:42679 every 8 connection(s)
I20260619 10:19:26.136410 825 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTU9OP6/build/release/bin/kudu as pid 13101
I20260619 10:19:26.136513 825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781864224577058-825-0/minicluster-data/ts-0/wal/instance
I20260619 10:19:26.136723 825 external_mini_cluster.cc:1428] Setting key 2c4c7afebcde5974748da4ec087333f0
Jun 19 10:19:26 dist-test-slave-6pt2 krb5kdc[12544](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864366, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.65@KRBTEST.COM for kudu/127.0.206.126@KRBTEST.COM
I20260619 10:19:26.145349 13234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:19:26.133313 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:43433 (local address 127.0.206.65:58961)
0619 10:19:26.133777 (+ 464us) negotiation.cc:107] Waiting for socket to connect
0619 10:19:26.133821 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0619 10:19:26.134473 (+ 652us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 10:19:26.134923 (+ 450us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 10:19:26.134928 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 10:19:26.135196 (+ 268us) client_negotiation.cc:189] Negotiated authn=SASL
0619 10:19:26.135574 (+ 378us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:19:26.135609 (+ 35us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:19:26.136755 (+ 1146us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:19:26.136756 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:19:26.138314 (+ 1558us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:19:26.138317 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:19:26.138882 (+ 565us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:19:26.138883 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:19:26.139018 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:19:26.139596 (+ 578us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 10:19:26.139607 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0619 10:19:26.141221 (+ 1614us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 10:19:26.143255 (+ 2034us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 10:19:26.143261 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 10:19:26.143263 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 10:19:26.143929 (+ 666us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 10:19:26.144199 (+ 270us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 10:19:26.144200 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 10:19:26.144200 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 10:19:26.144340 (+ 140us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 10:19:26.144666 (+ 326us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 10:19:26.144672 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 10:19:26.144844 (+ 172us) client_negotiation.cc:769] Sending connection context
0619 10:19:26.144973 (+ 129us) client_negotiation.cc:240] Negotiation successful
0619 10:19:26.145124 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":420,"thread_start_us":88,"threads_started":1}
I20260619 10:19:26.145903 13235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:19:26.133610 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.65:58961 (local address 127.0.206.126:43433)
0619 10:19:26.134683 (+ 1073us) server_negotiation.cc:206] Beginning negotiation
0619 10:19:26.134685 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 10:19:26.134694 (+ 9us) server_negotiation.cc:407] Connection header received
0619 10:19:26.134719 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:19:26.134719 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:19:26.134732 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:19:26.134797 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:19:26.135705 (+ 908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:19:26.136632 (+ 927us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:19:26.138412 (+ 1780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:19:26.138742 (+ 330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:19:26.138806 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:19:26.141324 (+ 2518us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 10:19:26.141325 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 10:19:26.141326 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 10:19:26.141332 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 10:19:26.143167 (+ 1835us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:19:26.144029 (+ 862us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:19:26.144030 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:19:26.144030 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:19:26.144118 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:19:26.144414 (+ 296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:19:26.144415 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:19:26.144415 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:19:26.144563 (+ 148us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 10:19:26.144629 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0619 10:19:26.145723 (+ 1094us) server_negotiation.cc:299] Negotiation successful
0619 10:19:26.145808 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1045,"thread_start_us":75,"threads_started":1}
I20260619 10:19:26.146701 13232 heartbeater.cc:347] Connected to a master server at 127.0.206.126:43433
I20260619 10:19:26.146799 13232 heartbeater.cc:464] Registering TS with master...
I20260619 10:19:26.146986 13232 heartbeater.cc:511] Master 127.0.206.126:43433 requested a full tablet report, sending...
I20260619 10:19:26.147536 13020 ts_manager.cc:194] Registered new tserver with Master: c31c3e7e16184ade9825d9bd931f460f (127.0.206.65:42679)
I20260619 10:19:26.148430 13020 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.206.65@KRBTEST.COM'} at 127.0.206.65:58961
WARNING: no policy specified for kudu/127.0.206.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.206.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.206.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781864224577058-825-0/krb5kdc/kudu_127.0.206.66.keytab.
Entry for principal kudu/127.0.206.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781864224577058-825-0/krb5kdc/kudu_127.0.206.66.keytab.
I20260619 10:19:26.181384 825 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTU9OP6/build/release/bin/kudu
/tmp/dist-test-taskTU9OP6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781864224577058-825-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781864224577058-825-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781864224577058-825-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781864224577058-825-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.0.206.84:59075/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.206.66:0
--local_ip_for_outbound_sockets=127.0.206.66
--webserver_interface=127.0.206.66
--webserver_port=0
--tserver_master_addrs=127.0.206.126:43433
--keytab_file=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781864224577058-825-0/krb5kdc/kudu_127.0.206.66.keytab
--principal=kudu/127.0.206.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.206.84:39201
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781864224577058-825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781864224577058-825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781864224577058-825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 10:19:26.297996 13238 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 10:19:26.298324 13238 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 10:19:26.298400 13238 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 10:19:26.300091 13238 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 10:19:26.300192 13238 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 10:19:26.300361 13238 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.206.66
I20260619 10:19:26.302379 13238 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.206.84:39201
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781864224577058-825-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781864224577058-825-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.0.206.84:59075/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781864224577058-825-0/krb5kdc/kudu_127.0.206.66.keytab
--principal=kudu/127.0.206.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.206.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781864224577058-825-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.206.66
--webserver_port=0
--tserver_master_addrs=127.0.206.126:43433
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13238
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.206.66
--log_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781864224577058-825-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 09:32:44 UTC on ccycloud.quasar-gbtldp.root.comops.site
build id 4455440
I20260619 10:19:26.302848 13238 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 10:19:26.303153 13238 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 10:19:26.309494 13246 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 10:19:26.309845 13245 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 10:19:26.310052 13248 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 10:19:26.310945 13238 server_base.cc:1034] running on GCE node
I20260619 10:19:26.311233 13238 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 10:19:26.311475 13238 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 10:19:26.312784 13238 hybrid_clock.cc:648] HybridClock initialized: now 1781864366312763 us; error 33 us; skew 500 ppm
Jun 19 10:19:26 dist-test-slave-6pt2 krb5kdc[12544](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864366, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 10:19:26.315828 13238 init.cc:377] Logged in from keytab as kudu/127.0.206.66@KRBTEST.COM (short username kudu)
I20260619 10:19:26.316313 13238 webserver.cc:460] Webserver started at http://127.0.206.66:44443/ using document root <none> and password file <none>
I20260619 10:19:26.316545 13238 fs_manager.cc:359] Metadata directory not provided
I20260619 10:19:26.316591 13238 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 10:19:26.316707 13238 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Jun 19 10:19:26 dist-test-slave-6pt2 krb5kdc[12544](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864366, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.66@KRBTEST.COM for HTTP/127.0.206.84@KRBTEST.COM
I20260619 10:19:26.333323 13238 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781864224577058-825-0/minicluster-data/ts-1/data/instance:
uuid: "3c405338bc8741858c843bf608832187"
format_stamp: "Formatted at 2026-06-19 10:19:26 on dist-test-slave-6pt2"
server_key: "c6c8259b7f92ab796c063189f59db863"
server_key_iv: "194e9d1fb23f5ecd1bb73414e05ed14b"
server_key_version: "kuduclusterkey@0"
I20260619 10:19:26.333762 13238 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781864224577058-825-0/minicluster-data/ts-1/wal/instance:
uuid: "3c405338bc8741858c843bf608832187"
format_stamp: "Formatted at 2026-06-19 10:19:26 on dist-test-slave-6pt2"
server_key: "c6c8259b7f92ab796c063189f59db863"
server_key_iv: "194e9d1fb23f5ecd1bb73414e05ed14b"
server_key_version: "kuduclusterkey@0"
I20260619 10:19:26.337661 13238 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.003s
Jun 19 10:19:26 dist-test-slave-6pt2 krb5kdc[12544](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1781864366, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 10:19:26.358227 13255 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 10:19:26.358402 13238 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260619 10:19:26.358496 13238 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781864224577058-825-0/minicluster-data/ts-1/data,/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781864224577058-825-0/minicluster-data/ts-1/wal
uuid: "3c405338bc8741858c843bf608832187"
format_stamp: "Formatted at 2026-06-19 10:19:26 on dist-test-slave-6pt2"
server_key: "c6c8259b7f92ab796c063189f59db863"
server_key_iv: "194e9d1fb23f5ecd1bb73414e05ed14b"
server_key_version: "kuduclusterkey@0"
I20260619 10:19:26.358592 13238 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781864224577058-825-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781864224577058-825-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781864224577058-825-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 10:19:26.375653 13238 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 10:19:26.378763 13238 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 10:19:26.378899 13238 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 10:19:26.379097 13238 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 10:19:26.379413 13238 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 10:19:26.379454 13238 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 10:19:26.379495 13238 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 10:19:26.379518 13238 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 10:19:26.385547 13238 rpc_server.cc:307] RPC server started. Bound to: 127.0.206.66:36861
I20260619 10:19:26.385936 13238 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781864224577058-825-0/minicluster-data/ts-1/data/info.pb
I20260619 10:19:26.387519 13368 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.206.66:36861 every 8 connection(s)
I20260619 10:19:26.387856 825 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTU9OP6/build/release/bin/kudu as pid 13238
I20260619 10:19:26.387951 825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781864224577058-825-0/minicluster-data/ts-1/wal/instance
I20260619 10:19:26.388142 825 external_mini_cluster.cc:1428] Setting key ece20fb155b88153462c1ba3dfb79249
Jun 19 10:19:26 dist-test-slave-6pt2 krb5kdc[12544](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864366, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.66@KRBTEST.COM for kudu/127.0.206.126@KRBTEST.COM
I20260619 10:19:26.413168 13235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:19:26.389319 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.66:35027 (local address 127.0.206.126:43433)
0619 10:19:26.389359 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0619 10:19:26.389361 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 10:19:26.394907 (+ 5546us) server_negotiation.cc:407] Connection header received
0619 10:19:26.403247 (+ 8340us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:19:26.403249 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:19:26.403264 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:19:26.403295 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:19:26.404165 (+ 870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:19:26.404919 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:19:26.406390 (+ 1471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:19:26.406710 (+ 320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:19:26.406780 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:19:26.409393 (+ 2613us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 10:19:26.409395 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 10:19:26.409395 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 10:19:26.409402 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 10:19:26.411638 (+ 2236us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:19:26.412123 (+ 485us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:19:26.412124 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:19:26.412125 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:19:26.412194 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:19:26.412458 (+ 264us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:19:26.412459 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:19:26.412459 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:19:26.412585 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 10:19:26.412624 (+ 39us) server_negotiation.cc:1035] Waiting for connection context
0619 10:19:26.412980 (+ 356us) server_negotiation.cc:299] Negotiation successful
0619 10:19:26.413069 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
WARNING: no policy specified for kudu/127.0.206.67@KRBTEST.COM; defaulting to no policy
I20260619 10:19:26.413910 13371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:19:26.389022 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:43433 (local address 127.0.206.66:35027)
0619 10:19:26.394702 (+ 5680us) negotiation.cc:107] Waiting for socket to connect
0619 10:19:26.394784 (+ 82us) client_negotiation.cc:174] Beginning negotiation
0619 10:19:26.403099 (+ 8315us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 10:19:26.403501 (+ 402us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 10:19:26.403506 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 10:19:26.403770 (+ 264us) client_negotiation.cc:189] Negotiated authn=SASL
0619 10:19:26.404041 (+ 271us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:19:26.404047 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:19:26.405032 (+ 985us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:19:26.405034 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:19:26.406288 (+ 1254us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:19:26.406290 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:19:26.406865 (+ 575us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:19:26.406867 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:19:26.407014 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:19:26.407629 (+ 615us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 10:19:26.407639 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0619 10:19:26.409289 (+ 1650us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 10:19:26.411743 (+ 2454us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 10:19:26.411749 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 10:19:26.411752 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 10:19:26.412033 (+ 281us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 10:19:26.412260 (+ 227us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 10:19:26.412260 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 10:19:26.412261 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 10:19:26.412380 (+ 119us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 10:19:26.412666 (+ 286us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 10:19:26.412672 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 10:19:26.412822 (+ 150us) client_negotiation.cc:769] Sending connection context
0619 10:19:26.413531 (+ 709us) client_negotiation.cc:240] Negotiation successful
0619 10:19:26.413686 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5623,"thread_start_us":98,"threads_started":1}
I20260619 10:19:26.414499 13369 heartbeater.cc:347] Connected to a master server at 127.0.206.126:43433
I20260619 10:19:26.414611 13369 heartbeater.cc:464] Registering TS with master...
I20260619 10:19:26.414791 13369 heartbeater.cc:511] Master 127.0.206.126:43433 requested a full tablet report, sending...
I20260619 10:19:26.415205 13020 ts_manager.cc:194] Registered new tserver with Master: 3c405338bc8741858c843bf608832187 (127.0.206.66:36861)
I20260619 10:19:26.415796 13020 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.206.66@KRBTEST.COM'} at 127.0.206.66:35027
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.206.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.206.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781864224577058-825-0/krb5kdc/kudu_127.0.206.67.keytab.
Entry for principal kudu/127.0.206.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781864224577058-825-0/krb5kdc/kudu_127.0.206.67.keytab.
I20260619 10:19:26.431946 825 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTU9OP6/build/release/bin/kudu
/tmp/dist-test-taskTU9OP6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781864224577058-825-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781864224577058-825-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781864224577058-825-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781864224577058-825-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.0.206.84:59075/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.206.67:0
--local_ip_for_outbound_sockets=127.0.206.67
--webserver_interface=127.0.206.67
--webserver_port=0
--tserver_master_addrs=127.0.206.126:43433
--keytab_file=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781864224577058-825-0/krb5kdc/kudu_127.0.206.67.keytab
--principal=kudu/127.0.206.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.206.84:39201
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781864224577058-825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781864224577058-825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781864224577058-825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 10:19:26.539937 13374 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 10:19:26.540170 13374 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 10:19:26.540210 13374 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 10:19:26.541903 13374 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 10:19:26.542021 13374 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 10:19:26.542100 13374 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.206.67
I20260619 10:19:26.544109 13374 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.206.84:39201
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781864224577058-825-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781864224577058-825-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.0.206.84:59075/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781864224577058-825-0/krb5kdc/kudu_127.0.206.67.keytab
--principal=kudu/127.0.206.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.206.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781864224577058-825-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.206.67
--webserver_port=0
--tserver_master_addrs=127.0.206.126:43433
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13374
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.206.67
--log_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781864224577058-825-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 09:32:44 UTC on ccycloud.quasar-gbtldp.root.comops.site
build id 4455440
I20260619 10:19:26.544350 13374 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 10:19:26.544588 13374 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 10:19:26.551326 13383 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 10:19:26.551927 13380 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 10:19:26.552116 13381 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 10:19:26.553205 13374 server_base.cc:1034] running on GCE node
I20260619 10:19:26.553431 13374 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 10:19:26.553740 13374 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 10:19:26.554937 13374 hybrid_clock.cc:648] HybridClock initialized: now 1781864366554912 us; error 38 us; skew 500 ppm
Jun 19 10:19:26 dist-test-slave-6pt2 krb5kdc[12544](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864366, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 10:19:26.557621 13374 init.cc:377] Logged in from keytab as kudu/127.0.206.67@KRBTEST.COM (short username kudu)
I20260619 10:19:26.558048 13374 webserver.cc:460] Webserver started at http://127.0.206.67:41779/ using document root <none> and password file <none>
I20260619 10:19:26.558305 13374 fs_manager.cc:359] Metadata directory not provided
I20260619 10:19:26.558369 13374 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 10:19:26.558504 13374 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Jun 19 10:19:26 dist-test-slave-6pt2 krb5kdc[12544](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864366, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.67@KRBTEST.COM for HTTP/127.0.206.84@KRBTEST.COM
I20260619 10:19:26.574955 13374 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781864224577058-825-0/minicluster-data/ts-2/data/instance:
uuid: "8a588c56d7f0465785e2844b19a876cd"
format_stamp: "Formatted at 2026-06-19 10:19:26 on dist-test-slave-6pt2"
server_key: "c5f74bdb0e1132516449abc1bf2d1af8"
server_key_iv: "acf77375ae5c95f4358495d97cbc1fc0"
server_key_version: "kuduclusterkey@0"
I20260619 10:19:26.575517 13374 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781864224577058-825-0/minicluster-data/ts-2/wal/instance:
uuid: "8a588c56d7f0465785e2844b19a876cd"
format_stamp: "Formatted at 2026-06-19 10:19:26 on dist-test-slave-6pt2"
server_key: "c5f74bdb0e1132516449abc1bf2d1af8"
server_key_iv: "acf77375ae5c95f4358495d97cbc1fc0"
server_key_version: "kuduclusterkey@0"
I20260619 10:19:26.583060 13374 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.001s sys 0.000s
I20260619 10:19:26.595728 13390 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 10:19:26.596611 13374 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 10:19:26.596735 13374 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781864224577058-825-0/minicluster-data/ts-2/data,/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781864224577058-825-0/minicluster-data/ts-2/wal
uuid: "8a588c56d7f0465785e2844b19a876cd"
format_stamp: "Formatted at 2026-06-19 10:19:26 on dist-test-slave-6pt2"
server_key: "c5f74bdb0e1132516449abc1bf2d1af8"
server_key_iv: "acf77375ae5c95f4358495d97cbc1fc0"
server_key_version: "kuduclusterkey@0"
I20260619 10:19:26.596841 13374 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781864224577058-825-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781864224577058-825-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781864224577058-825-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 10:19:26.632427 13374 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 10:19:26.634851 13374 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 10:19:26.635020 13374 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 10:19:26.635277 13374 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 10:19:26.635663 13374 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 10:19:26.635715 13374 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 10:19:26.635763 13374 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 10:19:26.635794 13374 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 10:19:26.642208 13374 rpc_server.cc:307] RPC server started. Bound to: 127.0.206.67:43705
I20260619 10:19:26.644537 13503 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.206.67:43705 every 8 connection(s)
I20260619 10:19:26.644727 13374 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781864224577058-825-0/minicluster-data/ts-2/data/info.pb
Jun 19 10:19:26 dist-test-slave-6pt2 krb5kdc[12544](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864366, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.67@KRBTEST.COM for kudu/127.0.206.126@KRBTEST.COM
I20260619 10:19:26.652870 825 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTU9OP6/build/release/bin/kudu as pid 13374
I20260619 10:19:26.652978 825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781864224577058-825-0/minicluster-data/ts-2/wal/instance
I20260619 10:19:26.653179 825 external_mini_cluster.cc:1428] Setting key efdd61f1243b187b4e6381eb950730d2
I20260619 10:19:26.654089 13235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:19:26.645773 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.67:45703 (local address 127.0.206.126:43433)
0619 10:19:26.645816 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0619 10:19:26.645818 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 10:19:26.645828 (+ 10us) server_negotiation.cc:407] Connection header received
0619 10:19:26.646450 (+ 622us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:19:26.646451 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:19:26.646465 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:19:26.646522 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:19:26.647189 (+ 667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:19:26.647694 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:19:26.648671 (+ 977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:19:26.648945 (+ 274us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:19:26.649035 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:19:26.651133 (+ 2098us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 10:19:26.651135 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 10:19:26.651136 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 10:19:26.651143 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 10:19:26.652496 (+ 1353us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:19:26.652988 (+ 492us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:19:26.652990 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:19:26.652990 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:19:26.653046 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:19:26.653327 (+ 281us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:19:26.653328 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:19:26.653328 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:19:26.653467 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 10:19:26.653530 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0619 10:19:26.653897 (+ 367us) server_negotiation.cc:299] Negotiation successful
0619 10:19:26.653992 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260619 10:19:26.654574 13506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:19:26.645517 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:43433 (local address 127.0.206.67:45703)
0619 10:19:26.645687 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0619 10:19:26.645729 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0619 10:19:26.646312 (+ 583us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 10:19:26.646611 (+ 299us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 10:19:26.646617 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 10:19:26.646886 (+ 269us) client_negotiation.cc:189] Negotiated authn=SASL
0619 10:19:26.647084 (+ 198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:19:26.647089 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:19:26.647789 (+ 700us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:19:26.647790 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:19:26.648580 (+ 790us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:19:26.648582 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:19:26.649056 (+ 474us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:19:26.649057 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:19:26.649177 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:19:26.649712 (+ 535us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 10:19:26.649720 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 10:19:26.651040 (+ 1320us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 10:19:26.652590 (+ 1550us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 10:19:26.652596 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 10:19:26.652598 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 10:19:26.652894 (+ 296us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 10:19:26.653123 (+ 229us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 10:19:26.653124 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 10:19:26.653125 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 10:19:26.653249 (+ 124us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 10:19:26.653580 (+ 331us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 10:19:26.653586 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 10:19:26.653746 (+ 160us) client_negotiation.cc:769] Sending connection context
0619 10:19:26.654244 (+ 498us) client_negotiation.cc:240] Negotiation successful
0619 10:19:26.654398 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":94,"threads_started":1}
I20260619 10:19:26.655040 13504 heartbeater.cc:347] Connected to a master server at 127.0.206.126:43433
I20260619 10:19:26.655133 13504 heartbeater.cc:464] Registering TS with master...
I20260619 10:19:26.655305 13504 heartbeater.cc:511] Master 127.0.206.126:43433 requested a full tablet report, sending...
I20260619 10:19:26.655689 13020 ts_manager.cc:194] Registered new tserver with Master: 8a588c56d7f0465785e2844b19a876cd (127.0.206.67:43705)
I20260619 10:19:26.656070 13020 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.206.67@KRBTEST.COM'} at 127.0.206.67:45703
I20260619 10:19:26.664933 825 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 10:19:26.666221 825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTU9OP6/build/release/bin/kudu with pid 13101
I20260619 10:19:26.671247 825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTU9OP6/build/release/bin/kudu with pid 13238
I20260619 10:19:26.675460 825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTU9OP6/build/release/bin/kudu with pid 13374
I20260619 10:19:26.684185 825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTU9OP6/build/release/bin/kudu with pid 12988
I20260619 10:19:26.690419 825 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTU9OP6/build/release/bin/kudu
/tmp/dist-test-taskTU9OP6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781864224577058-825-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781864224577058-825-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781864224577058-825-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781864224577058-825-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.206.126:43433
--webserver_interface=127.0.206.126
--webserver_port=39425
--keytab_file=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781864224577058-825-0/krb5kdc/kudu_127.0.206.126.keytab
--principal=kudu/127.0.206.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.206.84:39201
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.206.126:43433
--ranger_config_path=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781864224577058-825-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.0.206.84:59075/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.0.206.84:59075/kms with env {KRB5CCNAME=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781864224577058-825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781864224577058-825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781864224577058-825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 10:19:26.769284 13507 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 10:19:26.769471 13507 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 10:19:26.769493 13507 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 10:19:26.770670 13507 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 10:19:26.770709 13507 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 10:19:26.770725 13507 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 10:19:26.770780 13507 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 10:19:26.770797 13507 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 10:19:26.772294 13507 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.206.84:39201
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781864224577058-825-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781864224577058-825-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.0.206.84:59075/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.0.206.126:43433
--ranger_config_path=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781864224577058-825-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781864224577058-825-0/krb5kdc/kudu_127.0.206.126.keytab
--principal=kudu/127.0.206.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.206.126:43433
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781864224577058-825-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.206.126
--webserver_port=39425
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13507
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781864224577058-825-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 09:32:44 UTC on ccycloud.quasar-gbtldp.root.comops.site
build id 4455440
I20260619 10:19:26.772476 13507 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 10:19:26.772711 13507 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 10:19:26.777426 13515 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 10:19:26.777478 13512 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 10:19:26.777526 13507 server_base.cc:1034] running on GCE node
W20260619 10:19:26.777474 13513 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 10:19:26.777843 13507 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 10:19:26.778092 13507 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 10:19:26.779232 13507 hybrid_clock.cc:648] HybridClock initialized: now 1781864366779228 us; error 32 us; skew 500 ppm
Jun 19 10:19:26 dist-test-slave-6pt2 krb5kdc[12544](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864366, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 10:19:26.781454 13507 init.cc:377] Logged in from keytab as kudu/127.0.206.126@KRBTEST.COM (short username kudu)
I20260619 10:19:26.781890 13507 webserver.cc:460] Webserver started at http://127.0.206.126:39425/ using document root <none> and password file <none>
I20260619 10:19:26.782105 13507 fs_manager.cc:359] Metadata directory not provided
I20260619 10:19:26.782178 13507 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Jun 19 10:19:26 dist-test-slave-6pt2 krb5kdc[12544](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864366, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.126@KRBTEST.COM for HTTP/127.0.206.84@KRBTEST.COM
I20260619 10:19:26.852875 13507 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 10:19:26.853614 13523 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 10:19:26.853804 13507 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 10:19:26.853895 13507 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781864224577058-825-0/minicluster-data/master-0/data,/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781864224577058-825-0/minicluster-data/master-0/wal
uuid: "73991fccce974540bd47e4cfdc79aea7"
format_stamp: "Formatted at 2026-06-19 10:19:25 on dist-test-slave-6pt2"
server_key: "eb6f28be637b1dea25f38f297118f7f0"
server_key_iv: "8c31fcb84bac5de0c0679961690207df"
server_key_version: "kuduclusterkey@0"
I20260619 10:19:26.854260 13507 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781864224577058-825-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781864224577058-825-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781864224577058-825-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 10:19:26.887014 13507 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 10:19:26.888594 13507 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 10:19:26.888747 13507 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 10:19:26.893040 13507 rpc_server.cc:307] RPC server started. Bound to: 127.0.206.126:43433
I20260619 10:19:26.893100 13575 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.206.126:43433 every 8 connection(s)
I20260619 10:19:26.893469 13507 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781864224577058-825-0/minicluster-data/master-0/data/info.pb
I20260619 10:19:26.894665 13576 sys_catalog.cc:263] Verifying existing consensus state
I20260619 10:19:26.895316 13576 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 73991fccce974540bd47e4cfdc79aea7: Bootstrap starting.
I20260619 10:19:26.896957 825 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTU9OP6/build/release/bin/kudu as pid 13507
I20260619 10:19:26.897306 825 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTU9OP6/build/release/bin/kudu
/tmp/dist-test-taskTU9OP6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781864224577058-825-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781864224577058-825-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781864224577058-825-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781864224577058-825-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.0.206.84:59075/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.206.65:42679
--local_ip_for_outbound_sockets=127.0.206.65
--tserver_master_addrs=127.0.206.126:43433
--webserver_port=34033
--webserver_interface=127.0.206.65
--keytab_file=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781864224577058-825-0/krb5kdc/kudu_127.0.206.65.keytab
--principal=kudu/127.0.206.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.206.84:39201
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.0.206.84:59075/kms with env {KRB5CCNAME=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781864224577058-825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781864224577058-825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781864224577058-825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 10:19:26.903806 13576 log.cc:824] T 00000000000000000000000000000000 P 73991fccce974540bd47e4cfdc79aea7: Log is configured to *not* fsync() on all Append() calls
I20260619 10:19:26.905459 13576 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 73991fccce974540bd47e4cfdc79aea7: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260619 10:19:26.905743 13576 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 73991fccce974540bd47e4cfdc79aea7: Bootstrap complete.
I20260619 10:19:26.908082 13576 raft_consensus.cc:348] T 00000000000000000000000000000000 P 73991fccce974540bd47e4cfdc79aea7 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "73991fccce974540bd47e4cfdc79aea7" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 43433 } }
I20260619 10:19:26.908303 13576 raft_consensus.cc:729] T 00000000000000000000000000000000 P 73991fccce974540bd47e4cfdc79aea7 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 73991fccce974540bd47e4cfdc79aea7, State: Initialized, Role: FOLLOWER
I20260619 10:19:26.908471 13576 consensus_queue.cc:260] T 00000000000000000000000000000000 P 73991fccce974540bd47e4cfdc79aea7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "73991fccce974540bd47e4cfdc79aea7" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 43433 } }
I20260619 10:19:26.908535 13576 raft_consensus.cc:388] T 00000000000000000000000000000000 P 73991fccce974540bd47e4cfdc79aea7 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 10:19:26.908583 13576 raft_consensus.cc:482] T 00000000000000000000000000000000 P 73991fccce974540bd47e4cfdc79aea7 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 10:19:26.908628 13576 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 73991fccce974540bd47e4cfdc79aea7 [term 1 FOLLOWER]: Advancing to term 2
I20260619 10:19:26.909608 13576 raft_consensus.cc:504] T 00000000000000000000000000000000 P 73991fccce974540bd47e4cfdc79aea7 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "73991fccce974540bd47e4cfdc79aea7" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 43433 } }
I20260619 10:19:26.909726 13576 leader_election.cc:302] T 00000000000000000000000000000000 P 73991fccce974540bd47e4cfdc79aea7 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 73991fccce974540bd47e4cfdc79aea7; no voters:
I20260619 10:19:26.909920 13576 leader_election.cc:288] T 00000000000000000000000000000000 P 73991fccce974540bd47e4cfdc79aea7 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260619 10:19:26.909955 13580 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 73991fccce974540bd47e4cfdc79aea7 [term 2 FOLLOWER]: Leader election won for term 2
I20260619 10:19:26.910054 13580 raft_consensus.cc:686] T 00000000000000000000000000000000 P 73991fccce974540bd47e4cfdc79aea7 [term 2 LEADER]: Becoming Leader. State: Replica: 73991fccce974540bd47e4cfdc79aea7, State: Running, Role: LEADER
I20260619 10:19:26.910137 13580 consensus_queue.cc:237] T 00000000000000000000000000000000 P 73991fccce974540bd47e4cfdc79aea7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "73991fccce974540bd47e4cfdc79aea7" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 43433 } }
I20260619 10:19:26.910172 13576 sys_catalog.cc:564] T 00000000000000000000000000000000 P 73991fccce974540bd47e4cfdc79aea7 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 10:19:26.910650 13582 sys_catalog.cc:455] T 00000000000000000000000000000000 P 73991fccce974540bd47e4cfdc79aea7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 73991fccce974540bd47e4cfdc79aea7. Latest consensus state: current_term: 2 leader_uuid: "73991fccce974540bd47e4cfdc79aea7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "73991fccce974540bd47e4cfdc79aea7" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 43433 } } }
I20260619 10:19:26.910712 13581 sys_catalog.cc:455] T 00000000000000000000000000000000 P 73991fccce974540bd47e4cfdc79aea7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "73991fccce974540bd47e4cfdc79aea7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "73991fccce974540bd47e4cfdc79aea7" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 43433 } } }
I20260619 10:19:26.910744 13582 sys_catalog.cc:458] T 00000000000000000000000000000000 P 73991fccce974540bd47e4cfdc79aea7 [sys.catalog]: This master's current role is: LEADER
I20260619 10:19:26.910769 13581 sys_catalog.cc:458] T 00000000000000000000000000000000 P 73991fccce974540bd47e4cfdc79aea7 [sys.catalog]: This master's current role is: LEADER
I20260619 10:19:26.910933 13584 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 10:19:26.911409 13584 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 10:19:26.911547 13584 catalog_manager.cc:1259] Loaded cluster ID: 52c9af318b874300ba817d9f016418d4
I20260619 10:19:26.911595 13584 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 10:19:26.912016 13576 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781864224577058-825-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260619 10:19:26.915208 13584 catalog_manager.cc:1504] Loading token signing keys...
I20260619 10:19:26.915769 13584 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 73991fccce974540bd47e4cfdc79aea7: Loaded TSK: 0
I20260619 10:19:26.916075 13584 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260619 10:19:27.012408 13578 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 10:19:27.012681 13578 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 10:19:27.012712 13578 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 10:19:27.014374 13578 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 10:19:27.014424 13578 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 10:19:27.014460 13578 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.206.65
I20260619 10:19:27.016546 13578 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.206.84:39201
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781864224577058-825-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781864224577058-825-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.0.206.84:59075/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781864224577058-825-0/krb5kdc/kudu_127.0.206.65.keytab
--principal=kudu/127.0.206.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.206.65:42679
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781864224577058-825-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.206.65
--webserver_port=34033
--tserver_master_addrs=127.0.206.126:43433
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13578
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.206.65
--log_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781864224577058-825-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 09:32:44 UTC on ccycloud.quasar-gbtldp.root.comops.site
build id 4455440
I20260619 10:19:27.016778 13578 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 10:19:27.017038 13578 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 10:19:27.023064 13616 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 10:19:27.023319 13613 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 10:19:27.023761 13614 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 10:19:27.024050 13578 server_base.cc:1034] running on GCE node
I20260619 10:19:27.024361 13578 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 10:19:27.024641 13578 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 10:19:27.025854 13578 hybrid_clock.cc:648] HybridClock initialized: now 1781864367025827 us; error 43 us; skew 500 ppm
Jun 19 10:19:27 dist-test-slave-6pt2 krb5kdc[12544](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864367, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 10:19:27.029127 13578 init.cc:377] Logged in from keytab as kudu/127.0.206.65@KRBTEST.COM (short username kudu)
I20260619 10:19:27.029631 13578 webserver.cc:460] Webserver started at http://127.0.206.65:34033/ using document root <none> and password file <none>
I20260619 10:19:27.029879 13578 fs_manager.cc:359] Metadata directory not provided
I20260619 10:19:27.029956 13578 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Jun 19 10:19:27 dist-test-slave-6pt2 krb5kdc[12544](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864367, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.65@KRBTEST.COM for HTTP/127.0.206.84@KRBTEST.COM
I20260619 10:19:27.048624 13578 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 10:19:27.049574 13624 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 10:19:27.049712 13578 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 10:19:27.049772 13578 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781864224577058-825-0/minicluster-data/ts-0/data,/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781864224577058-825-0/minicluster-data/ts-0/wal
uuid: "c31c3e7e16184ade9825d9bd931f460f"
format_stamp: "Formatted at 2026-06-19 10:19:26 on dist-test-slave-6pt2"
server_key: "066650d496f4735e5ea78ec6225919da"
server_key_iv: "0d3cb177a7961e0632b747e2bdd25359"
server_key_version: "kuduclusterkey@0"
I20260619 10:19:27.050207 13578 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781864224577058-825-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781864224577058-825-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781864224577058-825-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 10:19:27.072408 13578 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 10:19:27.074437 13578 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 10:19:27.074640 13578 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 10:19:27.074903 13578 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 10:19:27.075284 13578 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 10:19:27.075335 13578 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 10:19:27.075373 13578 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 10:19:27.075398 13578 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 10:19:27.082283 13578 rpc_server.cc:307] RPC server started. Bound to: 127.0.206.65:42679
I20260619 10:19:27.082552 13737 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.206.65:42679 every 8 connection(s)
I20260619 10:19:27.082711 13578 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781864224577058-825-0/minicluster-data/ts-0/data/info.pb
I20260619 10:19:27.082995 825 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTU9OP6/build/release/bin/kudu as pid 13578
I20260619 10:19:27.083273 825 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTU9OP6/build/release/bin/kudu
/tmp/dist-test-taskTU9OP6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781864224577058-825-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781864224577058-825-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781864224577058-825-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781864224577058-825-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.0.206.84:59075/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.206.66:36861
--local_ip_for_outbound_sockets=127.0.206.66
--tserver_master_addrs=127.0.206.126:43433
--webserver_port=44443
--webserver_interface=127.0.206.66
--keytab_file=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781864224577058-825-0/krb5kdc/kudu_127.0.206.66.keytab
--principal=kudu/127.0.206.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.206.84:39201
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.0.206.84:59075/kms with env {KRB5CCNAME=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781864224577058-825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781864224577058-825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781864224577058-825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Jun 19 10:19:27 dist-test-slave-6pt2 krb5kdc[12544](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864367, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.65@KRBTEST.COM for kudu/127.0.206.126@KRBTEST.COM
I20260619 10:19:27.094992 13742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:19:27.084786 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.65:51669 (local address 127.0.206.126:43433)
0619 10:19:27.085891 (+ 1105us) server_negotiation.cc:206] Beginning negotiation
0619 10:19:27.085899 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 10:19:27.085928 (+ 29us) server_negotiation.cc:407] Connection header received
0619 10:19:27.086037 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:19:27.086042 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:19:27.086090 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:19:27.086228 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:19:27.086868 (+ 640us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:19:27.087823 (+ 955us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:19:27.088811 (+ 988us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:19:27.089224 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:19:27.089339 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:19:27.091553 (+ 2214us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 10:19:27.091557 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 10:19:27.091557 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 10:19:27.091576 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0619 10:19:27.093289 (+ 1713us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:19:27.093715 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:19:27.093718 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:19:27.093719 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:19:27.093808 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:19:27.094057 (+ 249us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:19:27.094058 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:19:27.094058 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:19:27.094285 (+ 227us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 10:19:27.094374 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0619 10:19:27.094687 (+ 313us) server_negotiation.cc:299] Negotiation successful
0619 10:19:27.094791 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1053,"thread_start_us":87,"threads_started":1}
I20260619 10:19:27.095003 13741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:19:27.084664 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:43433 (local address 127.0.206.65:51669)
0619 10:19:27.085167 (+ 503us) negotiation.cc:107] Waiting for socket to connect
0619 10:19:27.085212 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0619 10:19:27.085719 (+ 507us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 10:19:27.086263 (+ 544us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 10:19:27.086266 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 10:19:27.086489 (+ 223us) client_negotiation.cc:189] Negotiated authn=SASL
0619 10:19:27.086755 (+ 266us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:19:27.086760 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:19:27.087921 (+ 1161us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:19:27.087923 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:19:27.088713 (+ 790us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:19:27.088715 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:19:27.089342 (+ 627us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:19:27.089344 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:19:27.089490 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:19:27.090001 (+ 511us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 10:19:27.090015 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0619 10:19:27.091432 (+ 1417us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 10:19:27.093397 (+ 1965us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 10:19:27.093404 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 10:19:27.093406 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 10:19:27.093619 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 10:19:27.093878 (+ 259us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 10:19:27.093879 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 10:19:27.093879 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 10:19:27.093989 (+ 110us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 10:19:27.094367 (+ 378us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 10:19:27.094373 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 10:19:27.094505 (+ 132us) client_negotiation.cc:769] Sending connection context
0619 10:19:27.094640 (+ 135us) client_negotiation.cc:240] Negotiation successful
0619 10:19:27.094791 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":452,"thread_start_us":78,"threads_started":1}
I20260619 10:19:27.095999 13738 heartbeater.cc:347] Connected to a master server at 127.0.206.126:43433
I20260619 10:19:27.096100 13738 heartbeater.cc:464] Registering TS with master...
I20260619 10:19:27.096272 13738 heartbeater.cc:511] Master 127.0.206.126:43433 requested a full tablet report, sending...
W20260619 10:19:27.097070 13738 heartbeater.cc:644] Failed to heartbeat to 127.0.206.126:43433 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260619 10:19:27.097150 13738 heartbeater.cc:464] Registering TS with master...
I20260619 10:19:27.097213 13738 heartbeater.cc:511] Master 127.0.206.126:43433 requested a full tablet report, sending...
I20260619 10:19:27.097491 13738 heartbeater.cc:464] Registering TS with master...
I20260619 10:19:27.097572 13738 heartbeater.cc:511] Master 127.0.206.126:43433 requested a full tablet report, sending...
W20260619 10:19:27.097851 13738 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260619 10:19:27.201375 13740 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 10:19:27.201660 13740 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 10:19:27.201700 13740 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 10:19:27.203452 13740 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 10:19:27.203512 13740 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 10:19:27.203553 13740 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.206.66
I20260619 10:19:27.205722 13740 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.206.84:39201
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781864224577058-825-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781864224577058-825-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.0.206.84:59075/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781864224577058-825-0/krb5kdc/kudu_127.0.206.66.keytab
--principal=kudu/127.0.206.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.206.66:36861
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781864224577058-825-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.206.66
--webserver_port=44443
--tserver_master_addrs=127.0.206.126:43433
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13740
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.206.66
--log_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781864224577058-825-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 09:32:44 UTC on ccycloud.quasar-gbtldp.root.comops.site
build id 4455440
I20260619 10:19:27.205969 13740 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 10:19:27.206238 13740 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 10:19:27.212554 13750 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 10:19:27.212795 13747 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 10:19:27.212795 13740 server_base.cc:1034] running on GCE node
W20260619 10:19:27.212817 13748 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 10:19:27.213275 13740 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 10:19:27.213554 13740 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 10:19:27.214684 13740 hybrid_clock.cc:648] HybridClock initialized: now 1781864367214653 us; error 43 us; skew 500 ppm
Jun 19 10:19:27 dist-test-slave-6pt2 krb5kdc[12544](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864367, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 10:19:27.217626 13740 init.cc:377] Logged in from keytab as kudu/127.0.206.66@KRBTEST.COM (short username kudu)
I20260619 10:19:27.218122 13740 webserver.cc:460] Webserver started at http://127.0.206.66:44443/ using document root <none> and password file <none>
I20260619 10:19:27.218372 13740 fs_manager.cc:359] Metadata directory not provided
I20260619 10:19:27.218475 13740 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Jun 19 10:19:27 dist-test-slave-6pt2 krb5kdc[12544](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864367, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.66@KRBTEST.COM for HTTP/127.0.206.84@KRBTEST.COM
I20260619 10:19:27.237545 13740 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 10:19:27.238554 13758 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 10:19:27.238724 13740 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 10:19:27.238804 13740 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781864224577058-825-0/minicluster-data/ts-1/data,/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781864224577058-825-0/minicluster-data/ts-1/wal
uuid: "3c405338bc8741858c843bf608832187"
format_stamp: "Formatted at 2026-06-19 10:19:26 on dist-test-slave-6pt2"
server_key: "c6c8259b7f92ab796c063189f59db863"
server_key_iv: "194e9d1fb23f5ecd1bb73414e05ed14b"
server_key_version: "kuduclusterkey@0"
I20260619 10:19:27.239255 13740 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781864224577058-825-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781864224577058-825-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781864224577058-825-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 10:19:27.249100 13740 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 10:19:27.251165 13740 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 10:19:27.251303 13740 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 10:19:27.251536 13740 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 10:19:27.251866 13740 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 10:19:27.251904 13740 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 10:19:27.251925 13740 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 10:19:27.251961 13740 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 10:19:27.259322 13740 rpc_server.cc:307] RPC server started. Bound to: 127.0.206.66:36861
I20260619 10:19:27.259393 13871 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.206.66:36861 every 8 connection(s)
I20260619 10:19:27.259696 13740 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781864224577058-825-0/minicluster-data/ts-1/data/info.pb
I20260619 10:19:27.259919 825 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTU9OP6/build/release/bin/kudu as pid 13740
I20260619 10:19:27.260190 825 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTU9OP6/build/release/bin/kudu
/tmp/dist-test-taskTU9OP6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781864224577058-825-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781864224577058-825-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781864224577058-825-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781864224577058-825-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.0.206.84:59075/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.206.67:43705
--local_ip_for_outbound_sockets=127.0.206.67
--tserver_master_addrs=127.0.206.126:43433
--webserver_port=41779
--webserver_interface=127.0.206.67
--keytab_file=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781864224577058-825-0/krb5kdc/kudu_127.0.206.67.keytab
--principal=kudu/127.0.206.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.206.84:39201
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.0.206.84:59075/kms with env {KRB5CCNAME=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781864224577058-825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781864224577058-825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781864224577058-825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Jun 19 10:19:27 dist-test-slave-6pt2 krb5kdc[12544](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864367, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.66@KRBTEST.COM for kudu/127.0.206.126@KRBTEST.COM
I20260619 10:19:27.274323 13742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:19:27.261597 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.66:51755 (local address 127.0.206.126:43433)
0619 10:19:27.261671 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0619 10:19:27.261673 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 10:19:27.261684 (+ 11us) server_negotiation.cc:407] Connection header received
0619 10:19:27.261713 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:19:27.261714 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:19:27.261727 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:19:27.261784 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:19:27.263875 (+ 2091us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:19:27.264757 (+ 882us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:19:27.266582 (+ 1825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:19:27.266963 (+ 381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:19:27.267013 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:19:27.269756 (+ 2743us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 10:19:27.269757 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 10:19:27.269758 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 10:19:27.269766 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 10:19:27.271594 (+ 1828us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:19:27.273223 (+ 1629us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:19:27.273225 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:19:27.273225 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:19:27.273297 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:19:27.273576 (+ 279us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:19:27.273577 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:19:27.273578 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:19:27.273724 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 10:19:27.273801 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0619 10:19:27.274111 (+ 310us) server_negotiation.cc:299] Negotiation successful
0619 10:19:27.274222 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260619 10:19:27.274433 13874 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:19:27.260661 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:43433 (local address 127.0.206.66:51755)
0619 10:19:27.260842 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0619 10:19:27.260889 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0619 10:19:27.261516 (+ 627us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 10:19:27.262776 (+ 1260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 10:19:27.262780 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 10:19:27.263452 (+ 672us) client_negotiation.cc:189] Negotiated authn=SASL
0619 10:19:27.263753 (+ 301us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:19:27.263760 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:19:27.264918 (+ 1158us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:19:27.264919 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:19:27.266424 (+ 1505us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:19:27.266427 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:19:27.267137 (+ 710us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:19:27.267138 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:19:27.267284 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:19:27.267857 (+ 573us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 10:19:27.267873 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0619 10:19:27.269642 (+ 1769us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 10:19:27.271691 (+ 2049us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 10:19:27.271697 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 10:19:27.271700 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 10:19:27.273138 (+ 1438us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 10:19:27.273379 (+ 241us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 10:19:27.273380 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 10:19:27.273380 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 10:19:27.273503 (+ 123us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 10:19:27.273798 (+ 295us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 10:19:27.273803 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 10:19:27.273956 (+ 153us) client_negotiation.cc:769] Sending connection context
0619 10:19:27.274071 (+ 115us) client_negotiation.cc:240] Negotiation successful
0619 10:19:27.274215 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":81,"threads_started":1}
I20260619 10:19:27.274850 13872 heartbeater.cc:347] Connected to a master server at 127.0.206.126:43433
I20260619 10:19:27.274958 13872 heartbeater.cc:464] Registering TS with master...
I20260619 10:19:27.275142 13872 heartbeater.cc:511] Master 127.0.206.126:43433 requested a full tablet report, sending...
W20260619 10:19:27.275836 13872 heartbeater.cc:644] Failed to heartbeat to 127.0.206.126:43433 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260619 10:19:27.275904 13872 heartbeater.cc:464] Registering TS with master...
I20260619 10:19:27.275956 13872 heartbeater.cc:511] Master 127.0.206.126:43433 requested a full tablet report, sending...
I20260619 10:19:27.276297 13872 heartbeater.cc:464] Registering TS with master...
I20260619 10:19:27.276386 13872 heartbeater.cc:511] Master 127.0.206.126:43433 requested a full tablet report, sending...
W20260619 10:19:27.276700 13872 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260619 10:19:27.363224 13875 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 10:19:27.363513 13875 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 10:19:27.363554 13875 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 10:19:27.364935 13875 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 10:19:27.364993 13875 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 10:19:27.365020 13875 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.206.67
I20260619 10:19:27.367121 13875 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.206.84:39201
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781864224577058-825-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781864224577058-825-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.0.206.84:59075/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781864224577058-825-0/krb5kdc/kudu_127.0.206.67.keytab
--principal=kudu/127.0.206.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.206.67:43705
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781864224577058-825-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.206.67
--webserver_port=41779
--tserver_master_addrs=127.0.206.126:43433
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13875
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.206.67
--log_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781864224577058-825-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 09:32:44 UTC on ccycloud.quasar-gbtldp.root.comops.site
build id 4455440
I20260619 10:19:27.367354 13875 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 10:19:27.367591 13875 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 10:19:27.372694 13883 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 10:19:27.372656 13881 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 10:19:27.373026 13880 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 10:19:27.373188 13875 server_base.cc:1034] running on GCE node
I20260619 10:19:27.373413 13875 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 10:19:27.373656 13875 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 10:19:27.374811 13875 hybrid_clock.cc:648] HybridClock initialized: now 1781864367374803 us; error 29 us; skew 500 ppm
Jun 19 10:19:27 dist-test-slave-6pt2 krb5kdc[12544](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864367, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 10:19:27.377502 13875 init.cc:377] Logged in from keytab as kudu/127.0.206.67@KRBTEST.COM (short username kudu)
I20260619 10:19:27.377983 13875 webserver.cc:460] Webserver started at http://127.0.206.67:41779/ using document root <none> and password file <none>
I20260619 10:19:27.378250 13875 fs_manager.cc:359] Metadata directory not provided
I20260619 10:19:27.378306 13875 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Jun 19 10:19:27 dist-test-slave-6pt2 krb5kdc[12544](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864367, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.67@KRBTEST.COM for HTTP/127.0.206.84@KRBTEST.COM
I20260619 10:19:27.395242 13875 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.001s sys 0.000s
I20260619 10:19:27.396379 13891 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 10:19:27.396602 13875 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 10:19:27.396703 13875 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781864224577058-825-0/minicluster-data/ts-2/data,/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781864224577058-825-0/minicluster-data/ts-2/wal
uuid: "8a588c56d7f0465785e2844b19a876cd"
format_stamp: "Formatted at 2026-06-19 10:19:26 on dist-test-slave-6pt2"
server_key: "c5f74bdb0e1132516449abc1bf2d1af8"
server_key_iv: "acf77375ae5c95f4358495d97cbc1fc0"
server_key_version: "kuduclusterkey@0"
I20260619 10:19:27.397207 13875 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781864224577058-825-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781864224577058-825-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781864224577058-825-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 10:19:27.417218 13875 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 10:19:27.419680 13875 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 10:19:27.419844 13875 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 10:19:27.420078 13875 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 10:19:27.420459 13875 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 10:19:27.420511 13875 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 10:19:27.420550 13875 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 10:19:27.420575 13875 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 10:19:27.427604 13875 rpc_server.cc:307] RPC server started. Bound to: 127.0.206.67:43705
I20260619 10:19:27.427678 14005 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.206.67:43705 every 8 connection(s)
I20260619 10:19:27.427991 13875 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781864224577058-825-0/minicluster-data/ts-2/data/info.pb
Jun 19 10:19:27 dist-test-slave-6pt2 krb5kdc[12544](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864367, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.67@KRBTEST.COM for kudu/127.0.206.126@KRBTEST.COM
I20260619 10:19:27.436143 825 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTU9OP6/build/release/bin/kudu as pid 13875
I20260619 10:19:27.439405 14008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:19:27.428726 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:43433 (local address 127.0.206.67:59041)
0619 10:19:27.428981 (+ 255us) negotiation.cc:107] Waiting for socket to connect
0619 10:19:27.429022 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0619 10:19:27.429519 (+ 497us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 10:19:27.429884 (+ 365us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 10:19:27.429888 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 10:19:27.430178 (+ 290us) client_negotiation.cc:189] Negotiated authn=SASL
0619 10:19:27.430446 (+ 268us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:19:27.430452 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:19:27.431377 (+ 925us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:19:27.431378 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:19:27.432701 (+ 1323us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:19:27.432704 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:19:27.433103 (+ 399us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:19:27.433104 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:19:27.433247 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:19:27.433837 (+ 590us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 10:19:27.433847 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0619 10:19:27.435269 (+ 1422us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 10:19:27.437707 (+ 2438us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 10:19:27.437716 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 10:19:27.437718 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 10:19:27.438021 (+ 303us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 10:19:27.438247 (+ 226us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 10:19:27.438248 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 10:19:27.438249 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 10:19:27.438394 (+ 145us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 10:19:27.438716 (+ 322us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 10:19:27.438722 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 10:19:27.438885 (+ 163us) client_negotiation.cc:769] Sending connection context
0619 10:19:27.439000 (+ 115us) client_negotiation.cc:240] Negotiation successful
0619 10:19:27.439169 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":215,"thread_start_us":73,"threads_started":1}
I20260619 10:19:27.440091 13742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:19:27.428729 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.67:59041 (local address 127.0.206.126:43433)
0619 10:19:27.428779 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0619 10:19:27.428781 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 10:19:27.429126 (+ 345us) server_negotiation.cc:407] Connection header received
0619 10:19:27.429651 (+ 525us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:19:27.429652 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:19:27.429666 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:19:27.429698 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:19:27.430552 (+ 854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:19:27.431271 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:19:27.432811 (+ 1540us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:19:27.432992 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:19:27.433051 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:19:27.435393 (+ 2342us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 10:19:27.435395 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 10:19:27.435395 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 10:19:27.435403 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 10:19:27.437611 (+ 2208us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:19:27.438108 (+ 497us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:19:27.438109 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:19:27.438110 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:19:27.438169 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:19:27.438468 (+ 299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:19:27.438469 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:19:27.438470 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:19:27.438618 (+ 148us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 10:19:27.438685 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0619 10:19:27.439909 (+ 1224us) server_negotiation.cc:299] Negotiation successful
0619 10:19:27.439989 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260619 10:19:27.440474 14006 heartbeater.cc:347] Connected to a master server at 127.0.206.126:43433
I20260619 10:19:27.440572 14006 heartbeater.cc:464] Registering TS with master...
I20260619 10:19:27.440738 14006 heartbeater.cc:511] Master 127.0.206.126:43433 requested a full tablet report, sending...
W20260619 10:19:27.441416 14006 heartbeater.cc:644] Failed to heartbeat to 127.0.206.126:43433 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260619 10:19:27.441493 14006 heartbeater.cc:464] Registering TS with master...
I20260619 10:19:27.441560 14006 heartbeater.cc:511] Master 127.0.206.126:43433 requested a full tablet report, sending...
I20260619 10:19:27.441932 14006 heartbeater.cc:464] Registering TS with master...
I20260619 10:19:27.442019 14006 heartbeater.cc:511] Master 127.0.206.126:43433 requested a full tablet report, sending...
W20260619 10:19:27.442308 14006 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260619 10:19:27.449589 14010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:19:27.436719 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51310 (local address 127.0.206.126:43433)
0619 10:19:27.436984 (+ 265us) server_negotiation.cc:206] Beginning negotiation
0619 10:19:27.436985 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 10:19:27.436992 (+ 7us) server_negotiation.cc:407] Connection header received
0619 10:19:27.437007 (+ 15us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:19:27.437008 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:19:27.437039 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:19:27.437106 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:19:27.439434 (+ 2328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:19:27.440211 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:19:27.441303 (+ 1092us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:19:27.441556 (+ 253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:19:27.441618 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:19:27.443009 (+ 1391us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 10:19:27.443010 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 10:19:27.443010 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 10:19:27.443017 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 10:19:27.444704 (+ 1687us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:19:27.448851 (+ 4147us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:19:27.448853 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:19:27.448853 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:19:27.448916 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:19:27.449082 (+ 166us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:19:27.449083 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:19:27.449083 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:19:27.449205 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 10:19:27.449376 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0619 10:19:27.449423 (+ 47us) server_negotiation.cc:299] Negotiation successful
0619 10:19:27.449503 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":247,"thread_start_us":54,"threads_started":1}
Jun 19 10:19:27 dist-test-slave-6pt2 krb5kdc[12544](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1781864367, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 10:19:28.098048 13738 heartbeater.cc:464] Registering TS with master...
I20260619 10:19:28.098196 13738 heartbeater.cc:511] Master 127.0.206.126:43433 requested a full tablet report, sending...
I20260619 10:19:28.098652 13540 ts_manager.cc:194] Registered new tserver with Master: c31c3e7e16184ade9825d9bd931f460f (127.0.206.65:42679)
I20260619 10:19:28.099671 13540 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.206.65@KRBTEST.COM'} at 127.0.206.65:51669
I20260619 10:19:28.276875 13872 heartbeater.cc:464] Registering TS with master...
I20260619 10:19:28.277040 13872 heartbeater.cc:511] Master 127.0.206.126:43433 requested a full tablet report, sending...
I20260619 10:19:28.277446 13540 ts_manager.cc:194] Registered new tserver with Master: 3c405338bc8741858c843bf608832187 (127.0.206.66:36861)
I20260619 10:19:28.277967 13540 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.206.66@KRBTEST.COM'} at 127.0.206.66:51755
Jun 19 10:19:28 dist-test-slave-6pt2 krb5kdc[12544](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864367, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.126@KRBTEST.COM for HTTP/127.0.206.84@KRBTEST.COM
I20260619 10:19:28.442476 14006 heartbeater.cc:464] Registering TS with master...
I20260619 10:19:28.442649 14006 heartbeater.cc:511] Master 127.0.206.126:43433 requested a full tablet report, sending...
I20260619 10:19:28.442938 13540 ts_manager.cc:194] Registered new tserver with Master: 8a588c56d7f0465785e2844b19a876cd (127.0.206.67:43705)
I20260619 10:19:28.443499 13540 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.206.67@KRBTEST.COM'} at 127.0.206.67:59041
I20260619 10:19:28.451591 825 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 10:19:28.462517 14028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:19:28.452517 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51320 (local address 127.0.206.126:43433)
0619 10:19:28.454941 (+ 2424us) server_negotiation.cc:206] Beginning negotiation
0619 10:19:28.454943 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 10:19:28.454951 (+ 8us) server_negotiation.cc:407] Connection header received
0619 10:19:28.454973 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:19:28.454974 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:19:28.454984 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:19:28.455061 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:19:28.455734 (+ 673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:19:28.456493 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:19:28.457572 (+ 1079us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:19:28.457928 (+ 356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:19:28.458004 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:19:28.459493 (+ 1489us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 10:19:28.459494 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 10:19:28.459494 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 10:19:28.459502 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 10:19:28.461310 (+ 1808us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:19:28.461698 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:19:28.461699 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:19:28.461700 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:19:28.461765 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:19:28.461981 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:19:28.461982 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:19:28.461983 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:19:28.462117 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 10:19:28.462226 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0619 10:19:28.462334 (+ 108us) server_negotiation.cc:299] Negotiation successful
0619 10:19:28.462415 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2405,"thread_start_us":70,"threads_started":1}
I20260619 10:19:28.464658 13540 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51320:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 10:19:28.467036 13540 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 10:19:28.474571 14035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:19:28.469870 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.65:42679 (local address 127.0.0.1:36998)
0619 10:19:28.470283 (+ 413us) negotiation.cc:107] Waiting for socket to connect
0619 10:19:28.470320 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0619 10:19:28.470381 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 10:19:28.470834 (+ 453us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 10:19:28.470836 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 10:19:28.470838 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 10:19:28.470894 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:19:28.470895 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:19:28.472162 (+ 1267us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:19:28.472163 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:19:28.473383 (+ 1220us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:19:28.473385 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:19:28.474292 (+ 907us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:19:28.474293 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:19:28.474369 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:19:28.474382 (+ 13us) client_negotiation.cc:769] Sending connection context
0619 10:19:28.474450 (+ 68us) client_negotiation.cc:240] Negotiation successful
0619 10:19:28.474495 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":380,"thread_start_us":70,"threads_started":1}
I20260619 10:19:28.474884 14036 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:19:28.470118 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36998 (local address 127.0.206.65:42679)
0619 10:19:28.470495 (+ 377us) server_negotiation.cc:206] Beginning negotiation
0619 10:19:28.470499 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 10:19:28.470508 (+ 9us) server_negotiation.cc:407] Connection header received
0619 10:19:28.470542 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:19:28.470544 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:19:28.470559 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:19:28.470754 (+ 195us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 10:19:28.471009 (+ 255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:19:28.472076 (+ 1067us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:19:28.473476 (+ 1400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:19:28.474196 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:19:28.474286 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:19:28.474347 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0619 10:19:28.474477 (+ 130us) server_negotiation.cc:299] Negotiation successful
0619 10:19:28.474514 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":347,"thread_start_us":60,"threads_started":1}
I20260619 10:19:28.475607 13672 tablet_service.cc:1467] Processing CreateTablet for tablet 96351d165d5e4dfaaaabe66eaf4d2395 (DEFAULT_TABLE table=test-table [id=9525ce2c544b4b2d9cd8a9511f2ee428]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 10:19:28.475986 13672 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 96351d165d5e4dfaaaabe66eaf4d2395. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 10:19:28.479351 14035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:19:28.470087 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.67:43705 (local address 127.0.0.1:41208)
0619 10:19:28.475234 (+ 5147us) negotiation.cc:107] Waiting for socket to connect
0619 10:19:28.475244 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 10:19:28.475286 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 10:19:28.475526 (+ 240us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 10:19:28.475527 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 10:19:28.475528 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 10:19:28.475555 (+ 27us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:19:28.475556 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:19:28.476896 (+ 1340us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:19:28.476897 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:19:28.477974 (+ 1077us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:19:28.477976 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:19:28.479152 (+ 1176us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:19:28.479153 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:19:28.479211 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:19:28.479213 (+ 2us) client_negotiation.cc:769] Sending connection context
0619 10:19:28.479257 (+ 44us) client_negotiation.cc:240] Negotiation successful
0619 10:19:28.479289 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4839,"thread_start_us":66,"threads_started":1}
I20260619 10:19:28.479796 14037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:19:28.470173 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41208 (local address 127.0.206.67:43705)
0619 10:19:28.470703 (+ 530us) server_negotiation.cc:206] Beginning negotiation
0619 10:19:28.470707 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 10:19:28.475340 (+ 4633us) server_negotiation.cc:407] Connection header received
0619 10:19:28.475375 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:19:28.475377 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:19:28.475392 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:19:28.475454 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 10:19:28.475625 (+ 171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:19:28.476808 (+ 1183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:19:28.478076 (+ 1268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:19:28.478998 (+ 922us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:19:28.479078 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:19:28.479124 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0619 10:19:28.479631 (+ 507us) server_negotiation.cc:299] Negotiation successful
0619 10:19:28.479670 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":503,"thread_start_us":45,"threads_started":1}
I20260619 10:19:28.480397 14039 tablet_bootstrap.cc:492] T 96351d165d5e4dfaaaabe66eaf4d2395 P c31c3e7e16184ade9825d9bd931f460f: Bootstrap starting.
I20260619 10:19:28.480427 13940 tablet_service.cc:1467] Processing CreateTablet for tablet 96351d165d5e4dfaaaabe66eaf4d2395 (DEFAULT_TABLE table=test-table [id=9525ce2c544b4b2d9cd8a9511f2ee428]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 10:19:28.480778 13940 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 96351d165d5e4dfaaaabe66eaf4d2395. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 10:19:28.481036 14039 tablet_bootstrap.cc:654] T 96351d165d5e4dfaaaabe66eaf4d2395 P c31c3e7e16184ade9825d9bd931f460f: Neither blocks nor log segments found. Creating new log.
I20260619 10:19:28.481324 14039 log.cc:824] T 96351d165d5e4dfaaaabe66eaf4d2395 P c31c3e7e16184ade9825d9bd931f460f: Log is configured to *not* fsync() on all Append() calls
I20260619 10:19:28.482112 14039 tablet_bootstrap.cc:492] T 96351d165d5e4dfaaaabe66eaf4d2395 P c31c3e7e16184ade9825d9bd931f460f: No bootstrap required, opened a new log
I20260619 10:19:28.482195 14039 ts_tablet_manager.cc:1397] T 96351d165d5e4dfaaaabe66eaf4d2395 P c31c3e7e16184ade9825d9bd931f460f: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260619 10:19:28.485909 14039 raft_consensus.cc:348] T 96351d165d5e4dfaaaabe66eaf4d2395 P c31c3e7e16184ade9825d9bd931f460f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c405338bc8741858c843bf608832187" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 36861 } } peers { permanent_uuid: "c31c3e7e16184ade9825d9bd931f460f" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 42679 } } peers { permanent_uuid: "8a588c56d7f0465785e2844b19a876cd" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 43705 } }
I20260619 10:19:28.486069 14039 raft_consensus.cc:374] T 96351d165d5e4dfaaaabe66eaf4d2395 P c31c3e7e16184ade9825d9bd931f460f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 10:19:28.486107 14039 raft_consensus.cc:729] T 96351d165d5e4dfaaaabe66eaf4d2395 P c31c3e7e16184ade9825d9bd931f460f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c31c3e7e16184ade9825d9bd931f460f, State: Initialized, Role: FOLLOWER
I20260619 10:19:28.486236 14039 consensus_queue.cc:260] T 96351d165d5e4dfaaaabe66eaf4d2395 P c31c3e7e16184ade9825d9bd931f460f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c405338bc8741858c843bf608832187" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 36861 } } peers { permanent_uuid: "c31c3e7e16184ade9825d9bd931f460f" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 42679 } } peers { permanent_uuid: "8a588c56d7f0465785e2844b19a876cd" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 43705 } }
I20260619 10:19:28.486349 14034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:19:28.469873 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.66:36861 (local address 127.0.0.1:36828)
0619 10:19:28.470419 (+ 546us) negotiation.cc:107] Waiting for socket to connect
0619 10:19:28.470429 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 10:19:28.470774 (+ 345us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 10:19:28.482904 (+ 12130us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 10:19:28.482905 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 10:19:28.482908 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 10:19:28.482945 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:19:28.482946 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:19:28.484127 (+ 1181us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:19:28.484128 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:19:28.485196 (+ 1068us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:19:28.485198 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:19:28.486121 (+ 923us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:19:28.486122 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:19:28.486187 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:19:28.486191 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 10:19:28.486221 (+ 30us) client_negotiation.cc:240] Negotiation successful
0619 10:19:28.486279 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":530,"thread_start_us":80,"threads_started":1}
I20260619 10:19:28.486472 14039 ts_tablet_manager.cc:1428] T 96351d165d5e4dfaaaabe66eaf4d2395 P c31c3e7e16184ade9825d9bd931f460f: Time spent starting tablet: real 0.004s user 0.003s sys 0.000s
I20260619 10:19:28.486630 13738 heartbeater.cc:503] Master 127.0.206.126:43433 was elected leader, sending a full tablet report...
I20260619 10:19:28.486760 14038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:19:28.470277 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36828 (local address 127.0.206.66:36861)
0619 10:19:28.482751 (+ 12474us) server_negotiation.cc:206] Beginning negotiation
0619 10:19:28.482756 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 10:19:28.482768 (+ 12us) server_negotiation.cc:407] Connection header received
0619 10:19:28.482802 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:19:28.482803 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:19:28.482820 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:19:28.482906 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 10:19:28.483055 (+ 149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:19:28.484053 (+ 998us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:19:28.485293 (+ 1240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:19:28.486015 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:19:28.486497 (+ 482us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:19:28.486574 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0619 10:19:28.486623 (+ 49us) server_negotiation.cc:299] Negotiation successful
0619 10:19:28.486673 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12434,"thread_start_us":57,"threads_started":1}
I20260619 10:19:28.487334 13806 tablet_service.cc:1467] Processing CreateTablet for tablet 96351d165d5e4dfaaaabe66eaf4d2395 (DEFAULT_TABLE table=test-table [id=9525ce2c544b4b2d9cd8a9511f2ee428]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 10:19:28.487672 13806 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 96351d165d5e4dfaaaabe66eaf4d2395. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 10:19:28.487875 14041 tablet_bootstrap.cc:492] T 96351d165d5e4dfaaaabe66eaf4d2395 P 8a588c56d7f0465785e2844b19a876cd: Bootstrap starting.
I20260619 10:19:28.488448 14041 tablet_bootstrap.cc:654] T 96351d165d5e4dfaaaabe66eaf4d2395 P 8a588c56d7f0465785e2844b19a876cd: Neither blocks nor log segments found. Creating new log.
I20260619 10:19:28.488755 14041 log.cc:824] T 96351d165d5e4dfaaaabe66eaf4d2395 P 8a588c56d7f0465785e2844b19a876cd: Log is configured to *not* fsync() on all Append() calls
I20260619 10:19:28.489415 14041 tablet_bootstrap.cc:492] T 96351d165d5e4dfaaaabe66eaf4d2395 P 8a588c56d7f0465785e2844b19a876cd: No bootstrap required, opened a new log
I20260619 10:19:28.489483 14041 ts_tablet_manager.cc:1397] T 96351d165d5e4dfaaaabe66eaf4d2395 P 8a588c56d7f0465785e2844b19a876cd: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260619 10:19:28.491811 14041 raft_consensus.cc:348] T 96351d165d5e4dfaaaabe66eaf4d2395 P 8a588c56d7f0465785e2844b19a876cd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c405338bc8741858c843bf608832187" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 36861 } } peers { permanent_uuid: "c31c3e7e16184ade9825d9bd931f460f" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 42679 } } peers { permanent_uuid: "8a588c56d7f0465785e2844b19a876cd" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 43705 } }
I20260619 10:19:28.491959 14041 raft_consensus.cc:374] T 96351d165d5e4dfaaaabe66eaf4d2395 P 8a588c56d7f0465785e2844b19a876cd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 10:19:28.491990 14041 raft_consensus.cc:729] T 96351d165d5e4dfaaaabe66eaf4d2395 P 8a588c56d7f0465785e2844b19a876cd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a588c56d7f0465785e2844b19a876cd, State: Initialized, Role: FOLLOWER
I20260619 10:19:28.492092 14041 consensus_queue.cc:260] T 96351d165d5e4dfaaaabe66eaf4d2395 P 8a588c56d7f0465785e2844b19a876cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c405338bc8741858c843bf608832187" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 36861 } } peers { permanent_uuid: "c31c3e7e16184ade9825d9bd931f460f" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 42679 } } peers { permanent_uuid: "8a588c56d7f0465785e2844b19a876cd" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 43705 } }
I20260619 10:19:28.492304 14041 ts_tablet_manager.cc:1428] T 96351d165d5e4dfaaaabe66eaf4d2395 P 8a588c56d7f0465785e2844b19a876cd: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260619 10:19:28.492432 14006 heartbeater.cc:503] Master 127.0.206.126:43433 was elected leader, sending a full tablet report...
I20260619 10:19:28.492534 14044 tablet_bootstrap.cc:492] T 96351d165d5e4dfaaaabe66eaf4d2395 P 3c405338bc8741858c843bf608832187: Bootstrap starting.
I20260619 10:19:28.493451 14044 tablet_bootstrap.cc:654] T 96351d165d5e4dfaaaabe66eaf4d2395 P 3c405338bc8741858c843bf608832187: Neither blocks nor log segments found. Creating new log.
I20260619 10:19:28.493721 14044 log.cc:824] T 96351d165d5e4dfaaaabe66eaf4d2395 P 3c405338bc8741858c843bf608832187: Log is configured to *not* fsync() on all Append() calls
I20260619 10:19:28.495216 14044 tablet_bootstrap.cc:492] T 96351d165d5e4dfaaaabe66eaf4d2395 P 3c405338bc8741858c843bf608832187: No bootstrap required, opened a new log
I20260619 10:19:28.495277 14044 ts_tablet_manager.cc:1397] T 96351d165d5e4dfaaaabe66eaf4d2395 P 3c405338bc8741858c843bf608832187: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260619 10:19:28.497560 14044 raft_consensus.cc:348] T 96351d165d5e4dfaaaabe66eaf4d2395 P 3c405338bc8741858c843bf608832187 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c405338bc8741858c843bf608832187" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 36861 } } peers { permanent_uuid: "c31c3e7e16184ade9825d9bd931f460f" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 42679 } } peers { permanent_uuid: "8a588c56d7f0465785e2844b19a876cd" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 43705 } }
I20260619 10:19:28.497696 14044 raft_consensus.cc:374] T 96351d165d5e4dfaaaabe66eaf4d2395 P 3c405338bc8741858c843bf608832187 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 10:19:28.497727 14044 raft_consensus.cc:729] T 96351d165d5e4dfaaaabe66eaf4d2395 P 3c405338bc8741858c843bf608832187 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3c405338bc8741858c843bf608832187, State: Initialized, Role: FOLLOWER
I20260619 10:19:28.498196 14044 consensus_queue.cc:260] T 96351d165d5e4dfaaaabe66eaf4d2395 P 3c405338bc8741858c843bf608832187 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c405338bc8741858c843bf608832187" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 36861 } } peers { permanent_uuid: "c31c3e7e16184ade9825d9bd931f460f" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 42679 } } peers { permanent_uuid: "8a588c56d7f0465785e2844b19a876cd" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 43705 } }
I20260619 10:19:28.498528 14044 ts_tablet_manager.cc:1428] T 96351d165d5e4dfaaaabe66eaf4d2395 P 3c405338bc8741858c843bf608832187: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260619 10:19:28.498585 13872 heartbeater.cc:503] Master 127.0.206.126:43433 was elected leader, sending a full tablet report...
I20260619 10:19:28.502750 14042 raft_consensus.cc:482] T 96351d165d5e4dfaaaabe66eaf4d2395 P c31c3e7e16184ade9825d9bd931f460f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 10:19:28.502835 14042 raft_consensus.cc:504] T 96351d165d5e4dfaaaabe66eaf4d2395 P c31c3e7e16184ade9825d9bd931f460f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c405338bc8741858c843bf608832187" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 36861 } } peers { permanent_uuid: "c31c3e7e16184ade9825d9bd931f460f" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 42679 } } peers { permanent_uuid: "8a588c56d7f0465785e2844b19a876cd" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 43705 } }
I20260619 10:19:28.503151 14042 leader_election.cc:288] T 96351d165d5e4dfaaaabe66eaf4d2395 P c31c3e7e16184ade9825d9bd931f460f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3c405338bc8741858c843bf608832187 (127.0.206.66:36861), 8a588c56d7f0465785e2844b19a876cd (127.0.206.67:43705)
I20260619 10:19:28.507354 14048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:19:28.503332 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.66:36861 (local address 127.0.206.65:33997)
0619 10:19:28.503586 (+ 254us) negotiation.cc:107] Waiting for socket to connect
0619 10:19:28.503600 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0619 10:19:28.503654 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 10:19:28.504033 (+ 379us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 10:19:28.504034 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 10:19:28.504037 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 10:19:28.504130 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:19:28.504132 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:19:28.504947 (+ 815us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:19:28.504948 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:19:28.506421 (+ 1473us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:19:28.506423 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:19:28.507125 (+ 702us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:19:28.507126 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:19:28.507184 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:19:28.507187 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 10:19:28.507228 (+ 41us) client_negotiation.cc:240] Negotiation successful
0619 10:19:28.507264 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":223,"thread_start_us":68,"threads_started":1}
I20260619 10:19:28.507831 14038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:19:28.503719 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.65:33997 (local address 127.0.206.66:36861)
0619 10:19:28.503849 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0619 10:19:28.503852 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 10:19:28.503861 (+ 9us) server_negotiation.cc:407] Connection header received
0619 10:19:28.503886 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:19:28.503887 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:19:28.503899 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:19:28.503982 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 10:19:28.504208 (+ 226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:19:28.504854 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:19:28.506509 (+ 1655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:19:28.507019 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:19:28.507078 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:19:28.507085 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0619 10:19:28.507743 (+ 658us) server_negotiation.cc:299] Negotiation successful
0619 10:19:28.507779 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":110}
I20260619 10:19:28.508143 13826 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "96351d165d5e4dfaaaabe66eaf4d2395" candidate_uuid: "c31c3e7e16184ade9825d9bd931f460f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3c405338bc8741858c843bf608832187" is_pre_election: true
I20260619 10:19:28.508244 13826 raft_consensus.cc:2454] T 96351d165d5e4dfaaaabe66eaf4d2395 P 3c405338bc8741858c843bf608832187 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c31c3e7e16184ade9825d9bd931f460f in term 0.
I20260619 10:19:28.508412 13626 leader_election.cc:302] T 96351d165d5e4dfaaaabe66eaf4d2395 P c31c3e7e16184ade9825d9bd931f460f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c405338bc8741858c843bf608832187, c31c3e7e16184ade9825d9bd931f460f; no voters:
I20260619 10:19:28.508530 14042 raft_consensus.cc:2781] T 96351d165d5e4dfaaaabe66eaf4d2395 P c31c3e7e16184ade9825d9bd931f460f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 10:19:28.508561 14042 raft_consensus.cc:482] T 96351d165d5e4dfaaaabe66eaf4d2395 P c31c3e7e16184ade9825d9bd931f460f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 10:19:28.508579 14042 raft_consensus.cc:3037] T 96351d165d5e4dfaaaabe66eaf4d2395 P c31c3e7e16184ade9825d9bd931f460f [term 0 FOLLOWER]: Advancing to term 1
I20260619 10:19:28.508960 14037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:19:28.503655 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.65:41623 (local address 127.0.206.67:43705)
0619 10:19:28.503692 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0619 10:19:28.503694 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 10:19:28.503781 (+ 87us) server_negotiation.cc:407] Connection header received
0619 10:19:28.503874 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:19:28.503875 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:19:28.503885 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:19:28.503932 (+ 47us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 10:19:28.504172 (+ 240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:19:28.504856 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:19:28.508039 (+ 3183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:19:28.508591 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:19:28.508639 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:19:28.508646 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0619 10:19:28.508861 (+ 215us) server_negotiation.cc:299] Negotiation successful
0619 10:19:28.508899 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260619 10:19:28.509171 14049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:19:28.503480 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.67:43705 (local address 127.0.206.65:41623)
0619 10:19:28.503732 (+ 252us) negotiation.cc:107] Waiting for socket to connect
0619 10:19:28.503739 (+ 7us) client_negotiation.cc:174] Beginning negotiation
0619 10:19:28.503804 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 10:19:28.504000 (+ 196us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 10:19:28.504001 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 10:19:28.504003 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 10:19:28.504077 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:19:28.504080 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:19:28.504972 (+ 892us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:19:28.504973 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:19:28.507929 (+ 2956us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:19:28.507932 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:19:28.508705 (+ 773us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:19:28.508706 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:19:28.508776 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:19:28.508779 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 10:19:28.509058 (+ 279us) client_negotiation.cc:240] Negotiation successful
0619 10:19:28.509098 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":239,"mutex_wait_us":16,"thread_start_us":67,"threads_started":1}
I20260619 10:19:28.509338 14042 raft_consensus.cc:504] T 96351d165d5e4dfaaaabe66eaf4d2395 P c31c3e7e16184ade9825d9bd931f460f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c405338bc8741858c843bf608832187" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 36861 } } peers { permanent_uuid: "c31c3e7e16184ade9825d9bd931f460f" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 42679 } } peers { permanent_uuid: "8a588c56d7f0465785e2844b19a876cd" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 43705 } }
I20260619 10:19:28.509459 14042 leader_election.cc:288] T 96351d165d5e4dfaaaabe66eaf4d2395 P c31c3e7e16184ade9825d9bd931f460f [CANDIDATE]: Term 1 election: Requested vote from peers 3c405338bc8741858c843bf608832187 (127.0.206.66:36861), 8a588c56d7f0465785e2844b19a876cd (127.0.206.67:43705)
I20260619 10:19:28.509594 13960 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "96351d165d5e4dfaaaabe66eaf4d2395" candidate_uuid: "c31c3e7e16184ade9825d9bd931f460f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a588c56d7f0465785e2844b19a876cd" is_pre_election: true
I20260619 10:19:28.509703 13960 raft_consensus.cc:2454] T 96351d165d5e4dfaaaabe66eaf4d2395 P 8a588c56d7f0465785e2844b19a876cd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c31c3e7e16184ade9825d9bd931f460f in term 0.
I20260619 10:19:28.509698 13826 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "96351d165d5e4dfaaaabe66eaf4d2395" candidate_uuid: "c31c3e7e16184ade9825d9bd931f460f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3c405338bc8741858c843bf608832187"
I20260619 10:19:28.509760 13826 raft_consensus.cc:3037] T 96351d165d5e4dfaaaabe66eaf4d2395 P 3c405338bc8741858c843bf608832187 [term 0 FOLLOWER]: Advancing to term 1
I20260619 10:19:28.509799 13959 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "96351d165d5e4dfaaaabe66eaf4d2395" candidate_uuid: "c31c3e7e16184ade9825d9bd931f460f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a588c56d7f0465785e2844b19a876cd"
I20260619 10:19:28.509845 13959 raft_consensus.cc:3037] T 96351d165d5e4dfaaaabe66eaf4d2395 P 8a588c56d7f0465785e2844b19a876cd [term 0 FOLLOWER]: Advancing to term 1
I20260619 10:19:28.510538 13959 raft_consensus.cc:2454] T 96351d165d5e4dfaaaabe66eaf4d2395 P 8a588c56d7f0465785e2844b19a876cd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c31c3e7e16184ade9825d9bd931f460f in term 1.
I20260619 10:19:28.510545 13826 raft_consensus.cc:2454] T 96351d165d5e4dfaaaabe66eaf4d2395 P 3c405338bc8741858c843bf608832187 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c31c3e7e16184ade9825d9bd931f460f in term 1.
I20260619 10:19:28.510742 13626 leader_election.cc:302] T 96351d165d5e4dfaaaabe66eaf4d2395 P c31c3e7e16184ade9825d9bd931f460f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c405338bc8741858c843bf608832187, c31c3e7e16184ade9825d9bd931f460f; no voters:
I20260619 10:19:28.511089 14042 raft_consensus.cc:2781] T 96351d165d5e4dfaaaabe66eaf4d2395 P c31c3e7e16184ade9825d9bd931f460f [term 1 FOLLOWER]: Leader election won for term 1
I20260619 10:19:28.511245 14042 raft_consensus.cc:686] T 96351d165d5e4dfaaaabe66eaf4d2395 P c31c3e7e16184ade9825d9bd931f460f [term 1 LEADER]: Becoming Leader. State: Replica: c31c3e7e16184ade9825d9bd931f460f, State: Running, Role: LEADER
I20260619 10:19:28.511327 14042 consensus_queue.cc:237] T 96351d165d5e4dfaaaabe66eaf4d2395 P c31c3e7e16184ade9825d9bd931f460f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c405338bc8741858c843bf608832187" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 36861 } } peers { permanent_uuid: "c31c3e7e16184ade9825d9bd931f460f" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 42679 } } peers { permanent_uuid: "8a588c56d7f0465785e2844b19a876cd" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 43705 } }
I20260619 10:19:28.512001 13540 catalog_manager.cc:5495] T 96351d165d5e4dfaaaabe66eaf4d2395 P c31c3e7e16184ade9825d9bd931f460f reported cstate change: term changed from 0 to 1, leader changed from <none> to c31c3e7e16184ade9825d9bd931f460f (127.0.206.65). New cstate: current_term: 1 leader_uuid: "c31c3e7e16184ade9825d9bd931f460f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c405338bc8741858c843bf608832187" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 36861 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c31c3e7e16184ade9825d9bd931f460f" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 42679 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8a588c56d7f0465785e2844b19a876cd" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 43705 } health_report { overall_health: UNKNOWN } } }
W20260619 10:19:28.513216 13873 tablet.cc:2369] T 96351d165d5e4dfaaaabe66eaf4d2395 P 3c405338bc8741858c843bf608832187: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 10:19:28.521925 14036 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:19:28.518964 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37004 (local address 127.0.206.65:42679)
0619 10:19:28.519000 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0619 10:19:28.519002 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 10:19:28.519010 (+ 8us) server_negotiation.cc:407] Connection header received
0619 10:19:28.519042 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:19:28.519043 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:19:28.519060 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:19:28.519135 (+ 75us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 10:19:28.519261 (+ 126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:19:28.519965 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:19:28.521034 (+ 1069us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:19:28.521319 (+ 285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:19:28.521393 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:19:28.521585 (+ 192us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 10:19:28.521698 (+ 113us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 10:19:28.521789 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0619 10:19:28.521836 (+ 47us) server_negotiation.cc:299] Negotiation successful
0619 10:19:28.521856 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260619 10:19:28.523850 13959 raft_consensus.cc:1261] T 96351d165d5e4dfaaaabe66eaf4d2395 P 8a588c56d7f0465785e2844b19a876cd [term 1 FOLLOWER]: Refusing update from remote peer c31c3e7e16184ade9825d9bd931f460f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 10:19:28.523860 13826 raft_consensus.cc:1261] T 96351d165d5e4dfaaaabe66eaf4d2395 P 3c405338bc8741858c843bf608832187 [term 1 FOLLOWER]: Refusing update from remote peer c31c3e7e16184ade9825d9bd931f460f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 10:19:28.524168 14050 consensus_queue.cc:1035] T 96351d165d5e4dfaaaabe66eaf4d2395 P c31c3e7e16184ade9825d9bd931f460f [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a588c56d7f0465785e2844b19a876cd" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 43705 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 10:19:28.524320 14050 consensus_queue.cc:1035] T 96351d165d5e4dfaaaabe66eaf4d2395 P c31c3e7e16184ade9825d9bd931f460f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c405338bc8741858c843bf608832187" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 36861 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 10:19:28.529444 13540 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51320:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 10:19:28.529548 13540 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51320:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 10:19:28.530275 13540 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 73991fccce974540bd47e4cfdc79aea7: Sending DeleteTablet for 3 replicas of tablet 96351d165d5e4dfaaaabe66eaf4d2395
I20260619 10:19:28.530838 13672 tablet_service.cc:1514] Processing DeleteTablet for tablet 96351d165d5e4dfaaaabe66eaf4d2395 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 10:19:28 UTC) from {username='kudu', principal='kudu/127.0.206.126@KRBTEST.COM'} at 127.0.0.1:36998
I20260619 10:19:28.531329 825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTU9OP6/build/release/bin/kudu with pid 13578
I20260619 10:19:28.532632 14054 mvcc.cc:205] Tried to move back new op lower bound from 7298516453470420992 to 7298516453422043136. Current Snapshot: MvccSnapshot[applied={T|T < 7298516453470420992}]
I20260619 10:19:28.533188 13806 tablet_service.cc:1514] Processing DeleteTablet for tablet 96351d165d5e4dfaaaabe66eaf4d2395 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 10:19:28 UTC) from {username='kudu', principal='kudu/127.0.206.126@KRBTEST.COM'} at 127.0.0.1:36828
I20260619 10:19:28.533380 13940 tablet_service.cc:1514] Processing DeleteTablet for tablet 96351d165d5e4dfaaaabe66eaf4d2395 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 10:19:28 UTC) from {username='kudu', principal='kudu/127.0.206.126@KRBTEST.COM'} at 127.0.0.1:41208
I20260619 10:19:28.533579 14064 tablet_replica.cc:330] T 96351d165d5e4dfaaaabe66eaf4d2395 P 8a588c56d7f0465785e2844b19a876cd: stopping tablet replica
I20260619 10:19:28.533715 14064 raft_consensus.cc:2229] T 96351d165d5e4dfaaaabe66eaf4d2395 P 8a588c56d7f0465785e2844b19a876cd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 10:19:28.533872 14064 raft_consensus.cc:2258] T 96351d165d5e4dfaaaabe66eaf4d2395 P 8a588c56d7f0465785e2844b19a876cd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 10:19:28.536314 14063 tablet_replica.cc:330] T 96351d165d5e4dfaaaabe66eaf4d2395 P 3c405338bc8741858c843bf608832187: stopping tablet replica
I20260619 10:19:28.536412 14063 raft_consensus.cc:2229] T 96351d165d5e4dfaaaabe66eaf4d2395 P 3c405338bc8741858c843bf608832187 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 10:19:28.536505 14063 raft_consensus.cc:2258] T 96351d165d5e4dfaaaabe66eaf4d2395 P 3c405338bc8741858c843bf608832187 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 10:19:28.537284 14064 ts_tablet_manager.cc:1905] T 96351d165d5e4dfaaaabe66eaf4d2395 P 8a588c56d7f0465785e2844b19a876cd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 10:19:28.537320 14063 ts_tablet_manager.cc:1905] T 96351d165d5e4dfaaaabe66eaf4d2395 P 3c405338bc8741858c843bf608832187: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 10:19:28.538767 14064 ts_tablet_manager.cc:1918] T 96351d165d5e4dfaaaabe66eaf4d2395 P 8a588c56d7f0465785e2844b19a876cd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 10:19:28.538834 14064 log.cc:1196] T 96351d165d5e4dfaaaabe66eaf4d2395 P 8a588c56d7f0465785e2844b19a876cd: Deleting WAL directory at /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781864224577058-825-0/minicluster-data/ts-2/wal/wals/96351d165d5e4dfaaaabe66eaf4d2395
I20260619 10:19:28.539072 14064 ts_tablet_manager.cc:1939] T 96351d165d5e4dfaaaabe66eaf4d2395 P 8a588c56d7f0465785e2844b19a876cd: Deleting consensus metadata
W20260619 10:19:28.539044 13762 connection.cc:535] server connection from 127.0.206.65:33997 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260619 10:19:28.539366 13524 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.0.206.65:42679 (error 108)
I20260619 10:19:28.539422 13525 catalog_manager.cc:4841] TS 8a588c56d7f0465785e2844b19a876cd (127.0.206.67:43705): tablet 96351d165d5e4dfaaaabe66eaf4d2395 (table test-table [id=9525ce2c544b4b2d9cd8a9511f2ee428]) successfully deleted
I20260619 10:19:28.539965 14063 ts_tablet_manager.cc:1918] T 96351d165d5e4dfaaaabe66eaf4d2395 P 3c405338bc8741858c843bf608832187: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 10:19:28.540037 14063 log.cc:1196] T 96351d165d5e4dfaaaabe66eaf4d2395 P 3c405338bc8741858c843bf608832187: Deleting WAL directory at /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781864224577058-825-0/minicluster-data/ts-1/wal/wals/96351d165d5e4dfaaaabe66eaf4d2395
I20260619 10:19:28.540278 14063 ts_tablet_manager.cc:1939] T 96351d165d5e4dfaaaabe66eaf4d2395 P 3c405338bc8741858c843bf608832187: Deleting consensus metadata
I20260619 10:19:28.540692 13525 catalog_manager.cc:4841] TS 3c405338bc8741858c843bf608832187 (127.0.206.66:36861): tablet 96351d165d5e4dfaaaabe66eaf4d2395 (table test-table [id=9525ce2c544b4b2d9cd8a9511f2ee428]) successfully deleted
I20260619 10:19:28.540922 14034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:19:28.540793 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.65:42679 (local address 127.0.0.1:37008)
0619 10:19:28.540853 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0619 10:19:28.540895 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.65:42679: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260619 10:19:28.541029 13524 catalog_manager.cc:4584] TS c31c3e7e16184ade9825d9bd931f460f (127.0.206.65:42679): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 96351d165d5e4dfaaaabe66eaf4d2395: Network error: Client connection negotiation failed: client connection to 127.0.206.65:42679: connect: Connection refused (error 111)
I20260619 10:19:28.542855 825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTU9OP6/build/release/bin/kudu with pid 13740
I20260619 10:19:28.550197 825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTU9OP6/build/release/bin/kudu with pid 13875
I20260619 10:19:28.556325 825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTU9OP6/build/release/bin/kudu with pid 13507
I20260619 10:19:28.560720 825 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260619 10:19:28.895778 825 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260619 10:19:28.895860 825 mini_ranger.cc:66] Stopping Ranger...
I20260619 10:19:29.240569 825 mini_ranger.cc:68] Stopped Ranger
2026-06-19 10:19:29.240 UTC [12574] LOG: received smart shutdown request
2026-06-19 10:19:29.243 UTC [12574] LOG: background worker "logical replication launcher" (PID 12583) exited with exit code 1
2026-06-19 10:19:29.244 UTC [12578] LOG: shutting down
2026-06-19 10:19:29.387 UTC [12574] LOG: database system is shut down
2026-06-19T10:19:29Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (44936 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781864224577058-825-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 10:19:29 dist-test-slave-6pt2 krb5kdc[14074](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 10:19:29 dist-test-slave-6pt2 krb5kdc[14074](info): set up 2 sockets
Jun 19 10:19:29 dist-test-slave-6pt2 krb5kdc[14074](info): commencing operation
krb5kdc: starting...
W20260619 10:19:30.518455 825 mini_kdc.cc:121] Time spent starting KDC: real 1.002s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781864224577058-825-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781864224577058-825-0/krb5kdc/test-user.keytab.
Jun 19 10:19:30 dist-test-slave-6pt2 krb5kdc[14074](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864370, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T10:19:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T10:19:30Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.206.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.206.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.206.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781864224577058-825-0/krb5kdc/kudu_127.0.206.126.keytab.
Entry for principal kudu/127.0.206.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781864224577058-825-0/krb5kdc/kudu_127.0.206.126.keytab.
I20260619 10:19:30.627646 825 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTU9OP6/build/release/bin/kudu
/tmp/dist-test-taskTU9OP6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781864224577058-825-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781864224577058-825-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781864224577058-825-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781864224577058-825-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.206.126:44709
--webserver_interface=127.0.206.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781864224577058-825-0/krb5kdc/kudu_127.0.206.126.keytab
--principal=kudu/127.0.206.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.206.84:41877
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.206.126:44709,127.0.206.125:40605,127.0.206.124:39317
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781864224577058-825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781864224577058-825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781864224577058-825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 10:19:30.706287 14087 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 10:19:30.706491 14087 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 10:19:30.706513 14087 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 10:19:30.707691 14087 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 10:19:30.707732 14087 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 10:19:30.707746 14087 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 10:19:30.707799 14087 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 10:19:30.707824 14087 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 10:19:30.709295 14087 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.206.84:41877
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781864224577058-825-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781864224577058-825-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.206.126:44709,127.0.206.125:40605,127.0.206.124:39317
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781864224577058-825-0/krb5kdc/kudu_127.0.206.126.keytab
--principal=kudu/127.0.206.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.206.126:44709
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781864224577058-825-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.206.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14087
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781864224577058-825-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 09:32:44 UTC on ccycloud.quasar-gbtldp.root.comops.site
build id 4455440
I20260619 10:19:30.709549 14087 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 10:19:30.709784 14087 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 10:19:30.714452 14093 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 10:19:30.714464 14095 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 10:19:30.714464 14092 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 10:19:30.714546 14087 server_base.cc:1034] running on GCE node
I20260619 10:19:30.715008 14087 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 10:19:30.715301 14087 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 10:19:30.716459 14087 hybrid_clock.cc:648] HybridClock initialized: now 1781864370716437 us; error 56 us; skew 500 ppm
Jun 19 10:19:30 dist-test-slave-6pt2 krb5kdc[14074](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864370, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 10:19:30.721280 14087 init.cc:377] Logged in from keytab as kudu/127.0.206.126@KRBTEST.COM (short username kudu)
I20260619 10:19:30.721732 14087 webserver.cc:460] Webserver started at http://127.0.206.126:43425/ using document root <none> and password file <none>
I20260619 10:19:30.721946 14087 fs_manager.cc:359] Metadata directory not provided
I20260619 10:19:30.722009 14087 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 10:19:30.722157 14087 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 10:19:30.723227 14087 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781864224577058-825-0/minicluster-data/master-0/data/instance:
uuid: "9de25348fd9f407b8bc3872ba204e794"
format_stamp: "Formatted at 2026-06-19 10:19:30 on dist-test-slave-6pt2"
server_key: "ff7fec9e102f9f0cabb9d380cd0c34cf"
server_key_iv: "e1d0ab92112becc9bf11dca557697bc7"
server_key_version: "encryptionkey@0"
I20260619 10:19:30.723585 14087 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781864224577058-825-0/minicluster-data/master-0/wal/instance:
uuid: "9de25348fd9f407b8bc3872ba204e794"
format_stamp: "Formatted at 2026-06-19 10:19:30 on dist-test-slave-6pt2"
server_key: "ff7fec9e102f9f0cabb9d380cd0c34cf"
server_key_iv: "e1d0ab92112becc9bf11dca557697bc7"
server_key_version: "encryptionkey@0"
I20260619 10:19:30.724823 14087 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 10:19:30.725467 14102 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 10:19:30.725634 14087 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 10:19:30.725733 14087 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781864224577058-825-0/minicluster-data/master-0/data,/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781864224577058-825-0/minicluster-data/master-0/wal
uuid: "9de25348fd9f407b8bc3872ba204e794"
format_stamp: "Formatted at 2026-06-19 10:19:30 on dist-test-slave-6pt2"
server_key: "ff7fec9e102f9f0cabb9d380cd0c34cf"
server_key_iv: "e1d0ab92112becc9bf11dca557697bc7"
server_key_version: "encryptionkey@0"
I20260619 10:19:30.725809 14087 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781864224577058-825-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781864224577058-825-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781864224577058-825-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 10:19:30.744900 14087 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 10:19:30.746857 14087 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 10:19:30.747012 14087 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 10:19:30.750816 14087 rpc_server.cc:307] RPC server started. Bound to: 127.0.206.126:44709
I20260619 10:19:30.750862 14154 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.206.126:44709 every 8 connection(s)
I20260619 10:19:30.751210 14087 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781864224577058-825-0/minicluster-data/master-0/data/info.pb
I20260619 10:19:30.751976 825 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTU9OP6/build/release/bin/kudu as pid 14087
I20260619 10:19:30.752002 14155 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 10:19:30.752099 825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781864224577058-825-0/minicluster-data/master-0/wal/instance
I20260619 10:19:30.752283 825 external_mini_cluster.cc:1428] Setting key d555c6b43a05b5268193f9aae7261ee5
I20260619 10:19:30.753759 14155 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.206.126" port: 44709 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.0.206.125@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.206.125@KRBTEST.COM" created.
Jun 19 10:19:30 dist-test-slave-6pt2 krb5kdc[14074](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864370, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.126@KRBTEST.COM for kudu/127.0.206.126@KRBTEST.COM
I20260619 10:19:30.765429 14159 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:19:30.754644 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48954 (local address 127.0.206.126:44709)
0619 10:19:30.754850 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0619 10:19:30.754861 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0619 10:19:30.754938 (+ 77us) server_negotiation.cc:407] Connection header received
0619 10:19:30.755492 (+ 554us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:19:30.755494 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:19:30.755514 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:19:30.755613 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:19:30.756174 (+ 561us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:19:30.756983 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:19:30.757865 (+ 882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:19:30.758193 (+ 328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:19:30.758324 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:19:30.762194 (+ 3870us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 10:19:30.762202 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 10:19:30.762202 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 10:19:30.762210 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 10:19:30.763829 (+ 1619us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:19:30.764228 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:19:30.764231 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:19:30.764231 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:19:30.764352 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:19:30.764565 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:19:30.764566 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:19:30.764567 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:19:30.764776 (+ 209us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 10:19:30.764868 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0619 10:19:30.765055 (+ 187us) server_negotiation.cc:299] Negotiation successful
0619 10:19:30.765188 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":169,"thread_start_us":106,"threads_started":1}
I20260619 10:19:30.765431 14158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:19:30.754644 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:44709 (local address 127.0.0.1:48954)
0619 10:19:30.754817 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0619 10:19:30.754860 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0619 10:19:30.755335 (+ 475us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 10:19:30.755630 (+ 295us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 10:19:30.755632 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 10:19:30.755912 (+ 280us) client_negotiation.cc:189] Negotiated authn=SASL
0619 10:19:30.756064 (+ 152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:19:30.756068 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:19:30.757092 (+ 1024us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:19:30.757093 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:19:30.757769 (+ 676us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:19:30.757771 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:19:30.758295 (+ 524us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:19:30.758296 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:19:30.758359 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:19:30.758941 (+ 582us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 10:19:30.758949 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 10:19:30.762108 (+ 3159us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 10:19:30.763934 (+ 1826us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 10:19:30.763940 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 10:19:30.763942 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 10:19:30.764127 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 10:19:30.764414 (+ 287us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 10:19:30.764415 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 10:19:30.764415 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 10:19:30.764499 (+ 84us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 10:19:30.764849 (+ 350us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 10:19:30.764853 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 10:19:30.764913 (+ 60us) client_negotiation.cc:769] Sending connection context
0619 10:19:30.765017 (+ 104us) client_negotiation.cc:240] Negotiation successful
0619 10:19:30.765252 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"mutex_wait_us":104,"thread_start_us":80,"threads_started":1}
I20260619 10:19:30.766674 14155 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.206.125" port: 40605 } has no permanent_uuid. Determining permanent_uuid...
I20260619 10:19:30.767031 14158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:19:30.766892 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.125:40605 (local address 127.0.0.1:37788)
0619 10:19:30.766950 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0619 10:19:30.767001 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.125:40605: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260619 10:19:30.767180 14106 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.206.125:40605: connect: Connection refused (error 111)
I20260619 10:19:30.767643 14158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:19:30.767519 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.125:40605 (local address 127.0.0.1:37794)
0619 10:19:30.767573 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0619 10:19:30.767611 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.125:40605: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260619 10:19:30.767755 14155 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.206.125:40605: Network error: Client connection negotiation failed: client connection to 127.0.206.125:40605: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.206.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781864224577058-825-0/krb5kdc/kudu_127.0.206.125.keytab.
Entry for principal kudu/127.0.206.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781864224577058-825-0/krb5kdc/kudu_127.0.206.125.keytab.
I20260619 10:19:30.774974 825 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTU9OP6/build/release/bin/kudu
/tmp/dist-test-taskTU9OP6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781864224577058-825-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781864224577058-825-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781864224577058-825-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781864224577058-825-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.206.125:40605
--webserver_interface=127.0.206.125
--webserver_port=0
--keytab_file=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781864224577058-825-0/krb5kdc/kudu_127.0.206.125.keytab
--principal=kudu/127.0.206.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.206.84:41877
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.206.126:44709,127.0.206.125:40605,127.0.206.124:39317
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781864224577058-825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781864224577058-825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781864224577058-825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 10:19:30.817015 14155 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.206.125" port: 40605 } attempt: 1
I20260619 10:19:30.817533 14158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:19:30.817386 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.125:40605 (local address 127.0.0.1:37804)
0619 10:19:30.817453 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0619 10:19:30.817500 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.125:40605: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
I20260619 10:19:30.817919 14158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:19:30.817827 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.125:40605 (local address 127.0.0.1:37808)
0619 10:19:30.817874 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0619 10:19:30.817901 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.125:40605: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260619 10:19:30.818045 14155 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.206.125:40605: Network error: Client connection negotiation failed: client connection to 127.0.206.125:40605: connect: Connection refused (error 111)
W20260619 10:19:30.853426 14162 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 10:19:30.853611 14162 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 10:19:30.853633 14162 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 10:19:30.854696 14162 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 10:19:30.854727 14162 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 10:19:30.854740 14162 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 10:19:30.854753 14162 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 10:19:30.854764 14162 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 10:19:30.856163 14162 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.206.84:41877
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781864224577058-825-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781864224577058-825-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.0.206.126:44709,127.0.206.125:40605,127.0.206.124:39317
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781864224577058-825-0/krb5kdc/kudu_127.0.206.125.keytab
--principal=kudu/127.0.206.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.206.125:40605
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781864224577058-825-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.0.206.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14162
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781864224577058-825-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 09:32:44 UTC on ccycloud.quasar-gbtldp.root.comops.site
build id 4455440
I20260619 10:19:30.856343 14162 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 10:19:30.856503 14162 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 10:19:30.861136 14168 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 10:19:30.861090 14170 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 10:19:30.861303 14167 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 10:19:30.861392 14162 server_base.cc:1034] running on GCE node
I20260619 10:19:30.861676 14162 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 10:19:30.862001 14162 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 10:19:30.863178 14162 hybrid_clock.cc:648] HybridClock initialized: now 1781864370863215 us; error 94 us; skew 500 ppm
Jun 19 10:19:30 dist-test-slave-6pt2 krb5kdc[14074](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864370, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 10:19:30.865614 14162 init.cc:377] Logged in from keytab as kudu/127.0.206.125@KRBTEST.COM (short username kudu)
I20260619 10:19:30.866061 14162 webserver.cc:460] Webserver started at http://127.0.206.125:38339/ using document root <none> and password file <none>
I20260619 10:19:30.866297 14162 fs_manager.cc:359] Metadata directory not provided
I20260619 10:19:30.866341 14162 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 10:19:30.866503 14162 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 10:19:30.867542 14162 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781864224577058-825-0/minicluster-data/master-1/data/instance:
uuid: "0a3241cf26ce49748ad65b39ae174525"
format_stamp: "Formatted at 2026-06-19 10:19:30 on dist-test-slave-6pt2"
server_key: "2510c17da3a047297b476eda341d6038"
server_key_iv: "c1566d06accc355c1c46b397d0997270"
server_key_version: "encryptionkey@0"
I20260619 10:19:30.867893 14162 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781864224577058-825-0/minicluster-data/master-1/wal/instance:
uuid: "0a3241cf26ce49748ad65b39ae174525"
format_stamp: "Formatted at 2026-06-19 10:19:30 on dist-test-slave-6pt2"
server_key: "2510c17da3a047297b476eda341d6038"
server_key_iv: "c1566d06accc355c1c46b397d0997270"
server_key_version: "encryptionkey@0"
I20260619 10:19:30.869160 14162 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260619 10:19:30.869825 14177 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 10:19:30.870047 14162 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260619 10:19:30.870136 14162 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781864224577058-825-0/minicluster-data/master-1/data,/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781864224577058-825-0/minicluster-data/master-1/wal
uuid: "0a3241cf26ce49748ad65b39ae174525"
format_stamp: "Formatted at 2026-06-19 10:19:30 on dist-test-slave-6pt2"
server_key: "2510c17da3a047297b476eda341d6038"
server_key_iv: "c1566d06accc355c1c46b397d0997270"
server_key_version: "encryptionkey@0"
I20260619 10:19:30.870190 14162 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781864224577058-825-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781864224577058-825-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781864224577058-825-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 10:19:30.886243 14155 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.206.125" port: 40605 } attempt: 2
I20260619 10:19:30.886670 14158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:19:30.886510 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.125:40605 (local address 127.0.0.1:37824)
0619 10:19:30.886569 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0619 10:19:30.886635 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.125:40605: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
I20260619 10:19:30.887022 14158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:19:30.886926 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.125:40605 (local address 127.0.0.1:37826)
0619 10:19:30.886968 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0619 10:19:30.887005 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.125:40605: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260619 10:19:30.887109 14155 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.206.125:40605: Network error: Client connection negotiation failed: client connection to 127.0.206.125:40605: connect: Connection refused (error 111)
I20260619 10:19:30.896250 14162 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 10:19:30.897971 14162 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 10:19:30.898077 14162 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 10:19:30.902005 14162 rpc_server.cc:307] RPC server started. Bound to: 127.0.206.125:40605
I20260619 10:19:30.902053 14229 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.206.125:40605 every 8 connection(s)
I20260619 10:19:30.902360 14162 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781864224577058-825-0/minicluster-data/master-1/data/info.pb
I20260619 10:19:30.903195 14230 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 10:19:30.904687 14230 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.206.126" port: 44709 } has no permanent_uuid. Determining permanent_uuid...
I20260619 10:19:30.909389 825 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTU9OP6/build/release/bin/kudu as pid 14162
I20260619 10:19:30.909489 825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781864224577058-825-0/minicluster-data/master-1/wal/instance
I20260619 10:19:30.909700 825 external_mini_cluster.cc:1428] Setting key 0f3aeb57898a6d03516d44f01e374a12
Jun 19 10:19:30 dist-test-slave-6pt2 krb5kdc[14074](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864370, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.125@KRBTEST.COM for kudu/127.0.206.126@KRBTEST.COM
I20260619 10:19:30.913359 14159 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:19:30.905329 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48958 (local address 127.0.206.126:44709)
0619 10:19:30.905376 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0619 10:19:30.905378 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 10:19:30.905564 (+ 186us) server_negotiation.cc:407] Connection header received
0619 10:19:30.905986 (+ 422us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:19:30.905987 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:19:30.905996 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:19:30.906051 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:19:30.906636 (+ 585us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:19:30.907106 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:19:30.908009 (+ 903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:19:30.908192 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:19:30.908241 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:19:30.910494 (+ 2253us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 10:19:30.910495 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 10:19:30.910496 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 10:19:30.910501 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0619 10:19:30.912002 (+ 1501us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:19:30.912470 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:19:30.912470 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:19:30.912471 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:19:30.912518 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:19:30.912789 (+ 271us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:19:30.912789 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:19:30.912790 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:19:30.912880 (+ 90us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 10:19:30.912928 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0619 10:19:30.913242 (+ 314us) server_negotiation.cc:299] Negotiation successful
0619 10:19:30.913300 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260619 10:19:30.913669 14232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:19:30.905289 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:44709 (local address 127.0.0.1:48958)
0619 10:19:30.905445 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0619 10:19:30.905484 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0619 10:19:30.905910 (+ 426us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 10:19:30.906123 (+ 213us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 10:19:30.906126 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 10:19:30.906352 (+ 226us) client_negotiation.cc:189] Negotiated authn=SASL
0619 10:19:30.906530 (+ 178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:19:30.906533 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:19:30.907179 (+ 646us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:19:30.907180 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:19:30.907939 (+ 759us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:19:30.907941 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:19:30.908258 (+ 317us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:19:30.908258 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:19:30.908351 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:19:30.908756 (+ 405us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 10:19:30.908763 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0619 10:19:30.910400 (+ 1637us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 10:19:30.912080 (+ 1680us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 10:19:30.912086 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 10:19:30.912088 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 10:19:30.912390 (+ 302us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 10:19:30.912579 (+ 189us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 10:19:30.912580 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 10:19:30.912581 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 10:19:30.912728 (+ 147us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 10:19:30.912941 (+ 213us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 10:19:30.912947 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 10:19:30.913106 (+ 159us) client_negotiation.cc:769] Sending connection context
0619 10:19:30.913225 (+ 119us) client_negotiation.cc:240] Negotiation successful
0619 10:19:30.913397 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":87,"threads_started":1}
I20260619 10:19:30.914273 14230 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.206.125" port: 40605 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.0.206.124@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.206.124@KRBTEST.COM" created.
Jun 19 10:19:30 dist-test-slave-6pt2 krb5kdc[14074](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864370, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.125@KRBTEST.COM for kudu/127.0.206.125@KRBTEST.COM
I20260619 10:19:30.923879 14232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:19:30.914500 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.125:40605 (local address 127.0.0.1:37830)
0619 10:19:30.914543 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0619 10:19:30.914553 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 10:19:30.914592 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 10:19:30.914878 (+ 286us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 10:19:30.914879 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 10:19:30.914981 (+ 102us) client_negotiation.cc:189] Negotiated authn=SASL
0619 10:19:30.915022 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:19:30.915023 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:19:30.916309 (+ 1286us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:19:30.916310 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:19:30.916838 (+ 528us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:19:30.916839 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:19:30.917479 (+ 640us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:19:30.917480 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:19:30.917531 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:19:30.917889 (+ 358us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 10:19:30.917892 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0619 10:19:30.920848 (+ 2956us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 10:19:30.922862 (+ 2014us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 10:19:30.922863 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 10:19:30.922864 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 10:19:30.923014 (+ 150us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 10:19:30.923234 (+ 220us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 10:19:30.923235 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 10:19:30.923235 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 10:19:30.923316 (+ 81us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 10:19:30.923674 (+ 358us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 10:19:30.923675 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 10:19:30.923709 (+ 34us) client_negotiation.cc:769] Sending connection context
0619 10:19:30.923757 (+ 48us) client_negotiation.cc:240] Negotiation successful
0619 10:19:30.923821 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":28}
I20260619 10:19:30.924043 14234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:19:30.914577 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37830 (local address 127.0.206.125:40605)
0619 10:19:30.914719 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0619 10:19:30.914724 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 10:19:30.914745 (+ 21us) server_negotiation.cc:407] Connection header received
0619 10:19:30.914767 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:19:30.914769 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:19:30.914783 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:19:30.914858 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:19:30.915089 (+ 231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:19:30.916222 (+ 1133us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:19:30.916923 (+ 701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:19:30.917400 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:19:30.917485 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:19:30.920932 (+ 3447us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 10:19:30.920940 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 10:19:30.920940 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 10:19:30.920949 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0619 10:19:30.922771 (+ 1822us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:19:30.923107 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:19:30.923110 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:19:30.923110 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:19:30.923163 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:19:30.923403 (+ 240us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:19:30.923405 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:19:30.923406 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:19:30.923595 (+ 189us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 10:19:30.923681 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0619 10:19:30.923817 (+ 136us) server_negotiation.cc:299] Negotiation successful
0619 10:19:30.923899 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":113,"thread_start_us":89,"threads_started":1}
I20260619 10:19:30.924616 14230 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.206.124" port: 39317 } has no permanent_uuid. Determining permanent_uuid...
I20260619 10:19:30.924968 14232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:19:30.924836 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.124:39317 (local address 127.0.0.1:35058)
0619 10:19:30.924887 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0619 10:19:30.924935 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.124:39317: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260619 10:19:30.925112 14179 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.206.124:39317: connect: Connection refused (error 111)
I20260619 10:19:30.925555 14232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:19:30.925446 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.124:39317 (local address 127.0.0.1:35068)
0619 10:19:30.925499 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0619 10:19:30.925529 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.124:39317: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260619 10:19:30.925681 14230 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.206.124:39317: Network error: Client connection negotiation failed: client connection to 127.0.206.124:39317: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.206.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781864224577058-825-0/krb5kdc/kudu_127.0.206.124.keytab.
Entry for principal kudu/127.0.206.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781864224577058-825-0/krb5kdc/kudu_127.0.206.124.keytab.
I20260619 10:19:30.934181 825 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTU9OP6/build/release/bin/kudu
/tmp/dist-test-taskTU9OP6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781864224577058-825-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781864224577058-825-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781864224577058-825-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781864224577058-825-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.206.124:39317
--webserver_interface=127.0.206.124
--webserver_port=0
--keytab_file=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781864224577058-825-0/krb5kdc/kudu_127.0.206.124.keytab
--principal=kudu/127.0.206.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.206.84:41877
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.206.126:44709,127.0.206.125:40605,127.0.206.124:39317
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781864224577058-825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781864224577058-825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781864224577058-825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 10:19:30.974895 14230 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.206.124" port: 39317 } attempt: 1
I20260619 10:19:30.975373 14232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:19:30.975215 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.124:39317 (local address 127.0.0.1:35072)
0619 10:19:30.975288 (+ 73us) negotiation.cc:107] Waiting for socket to connect
0619 10:19:30.975335 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.124:39317: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
I20260619 10:19:30.975741 14232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:19:30.975651 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.124:39317 (local address 127.0.0.1:35086)
0619 10:19:30.975695 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0619 10:19:30.975722 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.124:39317: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260619 10:19:30.975862 14230 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.206.124:39317: Network error: Client connection negotiation failed: client connection to 127.0.206.124:39317: connect: Connection refused (error 111)
I20260619 10:19:30.978235 14155 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.206.125" port: 40605 } attempt: 3
Jun 19 10:19:30 dist-test-slave-6pt2 krb5kdc[14074](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864370, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.126@KRBTEST.COM for kudu/127.0.206.125@KRBTEST.COM
I20260619 10:19:30.985208 14158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:19:30.978505 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.125:40605 (local address 127.0.0.1:37834)
0619 10:19:30.978541 (+ 36us) negotiation.cc:107] Waiting for socket to connect
0619 10:19:30.978552 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 10:19:30.978637 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 10:19:30.978805 (+ 168us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 10:19:30.978806 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 10:19:30.978973 (+ 167us) client_negotiation.cc:189] Negotiated authn=SASL
0619 10:19:30.979034 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:19:30.979036 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:19:30.979724 (+ 688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:19:30.979725 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:19:30.980243 (+ 518us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:19:30.980244 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:19:30.980619 (+ 375us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:19:30.980620 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:19:30.980709 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:19:30.981202 (+ 493us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 10:19:30.981207 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 10:19:30.982553 (+ 1346us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 10:19:30.984119 (+ 1566us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 10:19:30.984120 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 10:19:30.984121 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 10:19:30.984299 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 10:19:30.984517 (+ 218us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 10:19:30.984518 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 10:19:30.984518 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 10:19:30.984578 (+ 60us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 10:19:30.984926 (+ 348us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 10:19:30.984927 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 10:19:30.984992 (+ 65us) client_negotiation.cc:769] Sending connection context
0619 10:19:30.985057 (+ 65us) client_negotiation.cc:240] Negotiation successful
0619 10:19:30.985134 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":16}
I20260619 10:19:30.985285 14234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:19:30.978556 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37834 (local address 127.0.206.125:40605)
0619 10:19:30.978633 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0619 10:19:30.978636 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 10:19:30.978647 (+ 11us) server_negotiation.cc:407] Connection header received
0619 10:19:30.978715 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:19:30.978716 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:19:30.978728 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:19:30.978799 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:19:30.979121 (+ 322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:19:30.979628 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:19:30.980324 (+ 696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:19:30.980543 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:19:30.980622 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:19:30.982710 (+ 2088us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 10:19:30.982711 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 10:19:30.982712 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 10:19:30.982718 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 10:19:30.984007 (+ 1289us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:19:30.984385 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:19:30.984386 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:19:30.984387 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:19:30.984444 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:19:30.984694 (+ 250us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:19:30.984696 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:19:30.984696 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:19:30.984847 (+ 151us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 10:19:30.984950 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0619 10:19:30.985108 (+ 158us) server_negotiation.cc:299] Negotiation successful
0619 10:19:30.985192 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260619 10:19:30.985630 14155 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.206.124" port: 39317 } has no permanent_uuid. Determining permanent_uuid...
I20260619 10:19:30.986012 14158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:19:30.985904 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.124:39317 (local address 127.0.0.1:35096)
0619 10:19:30.985954 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0619 10:19:30.985981 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.124:39317: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
I20260619 10:19:30.986357 14158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:19:30.986246 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.124:39317 (local address 127.0.0.1:35112)
0619 10:19:30.986289 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0619 10:19:30.986330 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.124:39317: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260619 10:19:30.986439 14155 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.206.124:39317: Network error: Client connection negotiation failed: client connection to 127.0.206.124:39317: connect: Connection refused (error 111)
W20260619 10:19:31.014887 14237 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 10:19:31.015139 14237 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 10:19:31.015175 14237 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 10:19:31.016283 14237 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 10:19:31.016348 14237 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 10:19:31.016374 14237 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 10:19:31.016397 14237 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 10:19:31.016426 14237 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 10:19:31.017563 14155 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.206.124" port: 39317 } attempt: 1
I20260619 10:19:31.018003 14158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:19:31.017852 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.124:39317 (local address 127.0.0.1:35128)
0619 10:19:31.017923 (+ 71us) negotiation.cc:107] Waiting for socket to connect
0619 10:19:31.017969 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.124:39317: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
I20260619 10:19:31.018054 14237 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.206.84:41877
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781864224577058-825-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781864224577058-825-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.0.206.126:44709,127.0.206.125:40605,127.0.206.124:39317
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781864224577058-825-0/krb5kdc/kudu_127.0.206.124.keytab
--principal=kudu/127.0.206.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.206.124:39317
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781864224577058-825-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.0.206.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14237
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781864224577058-825-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 09:32:44 UTC on ccycloud.quasar-gbtldp.root.comops.site
build id 4455440
I20260619 10:19:31.018296 14237 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 10:19:31.018379 14158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:19:31.018275 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.124:39317 (local address 127.0.0.1:35144)
0619 10:19:31.018307 (+ 32us) negotiation.cc:107] Waiting for socket to connect
0619 10:19:31.018356 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.124:39317: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":16}
W20260619 10:19:31.018466 14155 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.206.124:39317: Network error: Client connection negotiation failed: client connection to 127.0.206.124:39317: connect: Connection refused (error 111)
I20260619 10:19:31.018569 14237 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 10:19:31.023283 14243 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 10:19:31.023284 14242 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 10:19:31.023263 14245 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 10:19:31.023458 14237 server_base.cc:1034] running on GCE node
I20260619 10:19:31.023732 14237 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 10:19:31.024039 14237 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 10:19:31.025213 14237 hybrid_clock.cc:648] HybridClock initialized: now 1781864371025206 us; error 36 us; skew 500 ppm
Jun 19 10:19:31 dist-test-slave-6pt2 krb5kdc[14074](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864371, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 10:19:31.027683 14237 init.cc:377] Logged in from keytab as kudu/127.0.206.124@KRBTEST.COM (short username kudu)
I20260619 10:19:31.028144 14237 webserver.cc:460] Webserver started at http://127.0.206.124:33021/ using document root <none> and password file <none>
I20260619 10:19:31.028369 14237 fs_manager.cc:359] Metadata directory not provided
I20260619 10:19:31.028435 14237 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 10:19:31.028595 14237 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 10:19:31.029613 14237 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781864224577058-825-0/minicluster-data/master-2/data/instance:
uuid: "03f5aa499bf14e27821fc840af155fcb"
format_stamp: "Formatted at 2026-06-19 10:19:31 on dist-test-slave-6pt2"
server_key: "4bfc859e97d72d3f0a72257799f626ab"
server_key_iv: "d4ff8bccfbe52c414d7b8c1ffa125996"
server_key_version: "encryptionkey@0"
I20260619 10:19:31.029974 14237 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781864224577058-825-0/minicluster-data/master-2/wal/instance:
uuid: "03f5aa499bf14e27821fc840af155fcb"
format_stamp: "Formatted at 2026-06-19 10:19:31 on dist-test-slave-6pt2"
server_key: "4bfc859e97d72d3f0a72257799f626ab"
server_key_iv: "d4ff8bccfbe52c414d7b8c1ffa125996"
server_key_version: "encryptionkey@0"
I20260619 10:19:31.031316 14237 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260619 10:19:31.032012 14252 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 10:19:31.032179 14237 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260619 10:19:31.032282 14237 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781864224577058-825-0/minicluster-data/master-2/data,/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781864224577058-825-0/minicluster-data/master-2/wal
uuid: "03f5aa499bf14e27821fc840af155fcb"
format_stamp: "Formatted at 2026-06-19 10:19:31 on dist-test-slave-6pt2"
server_key: "4bfc859e97d72d3f0a72257799f626ab"
server_key_iv: "d4ff8bccfbe52c414d7b8c1ffa125996"
server_key_version: "encryptionkey@0"
I20260619 10:19:31.032366 14237 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781864224577058-825-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781864224577058-825-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781864224577058-825-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 10:19:31.041777 14237 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 10:19:31.044023 14237 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 10:19:31.044044 14230 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.206.124" port: 39317 } attempt: 2
I20260619 10:19:31.044212 14237 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 10:19:31.044497 14232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:19:31.044346 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.124:39317 (local address 127.0.0.1:35150)
0619 10:19:31.044416 (+ 70us) negotiation.cc:107] Waiting for socket to connect
0619 10:19:31.044459 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.124:39317: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
I20260619 10:19:31.044879 14232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:19:31.044770 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.124:39317 (local address 127.0.0.1:35154)
0619 10:19:31.044822 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0619 10:19:31.044855 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.124:39317: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260619 10:19:31.044992 14230 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.206.124:39317: Network error: Client connection negotiation failed: client connection to 127.0.206.124:39317: connect: Connection refused (error 111)
I20260619 10:19:31.048336 14237 rpc_server.cc:307] RPC server started. Bound to: 127.0.206.124:39317
I20260619 10:19:31.048365 14304 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.206.124:39317 every 8 connection(s)
I20260619 10:19:31.048700 14237 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781864224577058-825-0/minicluster-data/master-2/data/info.pb
I20260619 10:19:31.049480 14305 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 10:19:31.051011 14305 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.206.126" port: 44709 } has no permanent_uuid. Determining permanent_uuid...
Jun 19 10:19:31 dist-test-slave-6pt2 krb5kdc[14074](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864371, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.124@KRBTEST.COM for kudu/127.0.206.126@KRBTEST.COM
I20260619 10:19:31.058314 825 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTU9OP6/build/release/bin/kudu as pid 14237
I20260619 10:19:31.058398 825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781864224577058-825-0/minicluster-data/master-2/wal/instance
I20260619 10:19:31.058681 825 external_mini_cluster.cc:1428] Setting key 61d6afb4bdfd071520580f5db3dc0c81
I20260619 10:19:31.059984 14159 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:19:31.051629 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48960 (local address 127.0.206.126:44709)
0619 10:19:31.051722 (+ 93us) server_negotiation.cc:206] Beginning negotiation
0619 10:19:31.051725 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 10:19:31.051892 (+ 167us) server_negotiation.cc:407] Connection header received
0619 10:19:31.052391 (+ 499us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:19:31.052392 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:19:31.052405 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:19:31.052453 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:19:31.053017 (+ 564us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:19:31.053516 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:19:31.054447 (+ 931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:19:31.054696 (+ 249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:19:31.054780 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:19:31.056659 (+ 1879us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 10:19:31.056660 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 10:19:31.056660 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 10:19:31.056667 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 10:19:31.058515 (+ 1848us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:19:31.059023 (+ 508us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:19:31.059024 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:19:31.059024 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:19:31.059086 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:19:31.059366 (+ 280us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:19:31.059367 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:19:31.059367 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:19:31.059488 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 10:19:31.059542 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0619 10:19:31.059817 (+ 275us) server_negotiation.cc:299] Negotiation successful
0619 10:19:31.059895 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260619 10:19:31.060209 14307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:19:31.051589 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:44709 (local address 127.0.0.1:48960)
0619 10:19:31.051782 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0619 10:19:31.051814 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0619 10:19:31.052280 (+ 466us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 10:19:31.052520 (+ 240us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 10:19:31.052522 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 10:19:31.052746 (+ 224us) client_negotiation.cc:189] Negotiated authn=SASL
0619 10:19:31.052922 (+ 176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:19:31.052925 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:19:31.053608 (+ 683us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:19:31.053609 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:19:31.054355 (+ 746us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:19:31.054357 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:19:31.054788 (+ 431us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:19:31.054789 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:19:31.054908 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:19:31.055274 (+ 366us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 10:19:31.055282 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 10:19:31.056576 (+ 1294us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 10:19:31.058591 (+ 2015us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 10:19:31.058618 (+ 27us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 10:19:31.058621 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 10:19:31.058935 (+ 314us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 10:19:31.059171 (+ 236us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 10:19:31.059172 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 10:19:31.059172 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 10:19:31.059299 (+ 127us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 10:19:31.059568 (+ 269us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 10:19:31.059572 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 10:19:31.059681 (+ 109us) client_negotiation.cc:769] Sending connection context
0619 10:19:31.059793 (+ 112us) client_negotiation.cc:240] Negotiation successful
0619 10:19:31.059942 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":90,"threads_started":1}
I20260619 10:19:31.060726 14305 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.206.125" port: 40605 } has no permanent_uuid. Determining permanent_uuid...
Jun 19 10:19:31 dist-test-slave-6pt2 krb5kdc[14074](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864371, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.124@KRBTEST.COM for kudu/127.0.206.125@KRBTEST.COM
Jun 19 10:19:31 dist-test-slave-6pt2 krb5kdc[14074](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864370, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.206.124@KRBTEST.COM
I20260619 10:19:31.067986 14307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:19:31.060966 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.125:40605 (local address 127.0.0.1:37838)
0619 10:19:31.061016 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0619 10:19:31.061026 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 10:19:31.061060 (+ 34us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 10:19:31.061203 (+ 143us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 10:19:31.061203 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 10:19:31.061331 (+ 128us) client_negotiation.cc:189] Negotiated authn=SASL
0619 10:19:31.061375 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:19:31.061376 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:19:31.061993 (+ 617us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:19:31.061994 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:19:31.062511 (+ 517us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:19:31.062512 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:19:31.062905 (+ 393us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:19:31.062906 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:19:31.062968 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:19:31.063383 (+ 415us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 10:19:31.063386 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0619 10:19:31.064456 (+ 1070us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 10:19:31.066822 (+ 2366us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 10:19:31.066822 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 10:19:31.066823 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 10:19:31.067127 (+ 304us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 10:19:31.067357 (+ 230us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 10:19:31.067358 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 10:19:31.067358 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 10:19:31.067404 (+ 46us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 10:19:31.067698 (+ 294us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 10:19:31.067699 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 10:19:31.067729 (+ 30us) client_negotiation.cc:769] Sending connection context
0619 10:19:31.067814 (+ 85us) client_negotiation.cc:240] Negotiation successful
0619 10:19:31.067894 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36}
I20260619 10:19:31.067986 14234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:19:31.061035 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37838 (local address 127.0.206.125:40605)
0619 10:19:31.061100 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0619 10:19:31.061102 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 10:19:31.061111 (+ 9us) server_negotiation.cc:407] Connection header received
0619 10:19:31.061133 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:19:31.061134 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:19:31.061143 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:19:31.061195 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:19:31.061448 (+ 253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:19:31.061908 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:19:31.062630 (+ 722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:19:31.062824 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:19:31.062905 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:19:31.064551 (+ 1646us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 10:19:31.064553 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 10:19:31.064553 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 10:19:31.064561 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 10:19:31.066747 (+ 2186us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:19:31.067220 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:19:31.067221 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:19:31.067221 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:19:31.067294 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:19:31.067462 (+ 168us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:19:31.067463 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:19:31.067463 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:19:31.067608 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 10:19:31.067698 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0619 10:19:31.067843 (+ 145us) server_negotiation.cc:299] Negotiation successful
0619 10:19:31.067896 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260619 10:19:31.068563 14309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:19:31.059050 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35168 (local address 127.0.206.124:39317)
0619 10:19:31.059211 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0619 10:19:31.059215 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 10:19:31.059232 (+ 17us) server_negotiation.cc:407] Connection header received
0619 10:19:31.059275 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:19:31.059277 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:19:31.059290 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:19:31.059360 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:19:31.060190 (+ 830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:19:31.061187 (+ 997us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:19:31.061963 (+ 776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:19:31.062289 (+ 326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:19:31.062391 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:19:31.065345 (+ 2954us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 10:19:31.065350 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 10:19:31.065351 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 10:19:31.065356 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0619 10:19:31.067135 (+ 1779us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:19:31.067557 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:19:31.067559 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:19:31.067559 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:19:31.067602 (+ 43us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:19:31.067807 (+ 205us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:19:31.067807 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:19:31.067808 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:19:31.067958 (+ 150us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 10:19:31.068290 (+ 332us) server_negotiation.cc:1035] Waiting for connection context
0619 10:19:31.068355 (+ 65us) server_negotiation.cc:299] Negotiation successful
0619 10:19:31.068447 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":131,"thread_start_us":97,"threads_started":1}
I20260619 10:19:31.068586 14305 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.206.124" port: 39317 } has no permanent_uuid. Determining permanent_uuid...
Jun 19 10:19:31 dist-test-slave-6pt2 krb5kdc[14074](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864371, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.124@KRBTEST.COM for kudu/127.0.206.124@KRBTEST.COM
Jun 19 10:19:31 dist-test-slave-6pt2 krb5kdc[14074](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864370, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.206.126@KRBTEST.COM
I20260619 10:19:31.076833 14159 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:19:31.069495 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48968 (local address 127.0.206.126:44709)
0619 10:19:31.069534 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0619 10:19:31.069536 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 10:19:31.069588 (+ 52us) server_negotiation.cc:407] Connection header received
0619 10:19:31.069655 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:19:31.069655 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:19:31.069664 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:19:31.069717 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:19:31.070165 (+ 448us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:19:31.070591 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:19:31.071266 (+ 675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:19:31.071475 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:19:31.071568 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:19:31.073986 (+ 2418us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 10:19:31.073987 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 10:19:31.073987 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 10:19:31.073994 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 10:19:31.075718 (+ 1724us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:19:31.076137 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:19:31.076138 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:19:31.076138 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:19:31.076190 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:19:31.076405 (+ 215us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:19:31.076405 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:19:31.076406 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:19:31.076497 (+ 91us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 10:19:31.076634 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0619 10:19:31.076721 (+ 87us) server_negotiation.cc:299] Negotiation successful
0619 10:19:31.076775 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260619 10:19:31.077437 14307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:19:31.068911 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.124:39317 (local address 127.0.0.1:35172)
0619 10:19:31.069586 (+ 675us) negotiation.cc:107] Waiting for socket to connect
0619 10:19:31.069596 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 10:19:31.069646 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 10:19:31.069754 (+ 108us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 10:19:31.069754 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 10:19:31.069881 (+ 127us) client_negotiation.cc:189] Negotiated authn=SASL
0619 10:19:31.069902 (+ 21us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:19:31.069903 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:19:31.070489 (+ 586us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:19:31.070490 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:19:31.071015 (+ 525us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:19:31.071017 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:19:31.071370 (+ 353us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:19:31.071371 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:19:31.071419 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:19:31.071801 (+ 382us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 10:19:31.071806 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 10:19:31.073233 (+ 1427us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 10:19:31.075836 (+ 2603us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 10:19:31.075836 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 10:19:31.075837 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 10:19:31.075994 (+ 157us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 10:19:31.076708 (+ 714us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 10:19:31.076709 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 10:19:31.076710 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 10:19:31.076807 (+ 97us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 10:19:31.077115 (+ 308us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 10:19:31.077116 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 10:19:31.077162 (+ 46us) client_negotiation.cc:769] Sending connection context
0619 10:19:31.077248 (+ 86us) client_negotiation.cc:240] Negotiation successful
0619 10:19:31.077351 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":656}
I20260619 10:19:31.077586 14309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:19:31.068998 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35172 (local address 127.0.206.124:39317)
0619 10:19:31.069029 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0619 10:19:31.069031 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 10:19:31.069647 (+ 616us) server_negotiation.cc:407] Connection header received
0619 10:19:31.069699 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:19:31.069699 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:19:31.069708 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:19:31.069753 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:19:31.069969 (+ 216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:19:31.070415 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:19:31.071094 (+ 679us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:19:31.071291 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:19:31.071370 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:19:31.073319 (+ 1949us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 10:19:31.073320 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 10:19:31.073320 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 10:19:31.073326 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 10:19:31.075759 (+ 2433us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:19:31.076065 (+ 306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:19:31.076066 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:19:31.076066 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:19:31.076121 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:19:31.076886 (+ 765us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:19:31.076887 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:19:31.076888 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:19:31.077035 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 10:19:31.077120 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0619 10:19:31.077275 (+ 155us) server_negotiation.cc:299] Negotiation successful
0619 10:19:31.077446 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":100,"server-negotiator.queue_time_us":15}
I20260619 10:19:31.079022 14305 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 03f5aa499bf14e27821fc840af155fcb: Bootstrap starting.
I20260619 10:19:31.080015 14305 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 03f5aa499bf14e27821fc840af155fcb: Neither blocks nor log segments found. Creating new log.
I20260619 10:19:31.080387 14305 log.cc:824] T 00000000000000000000000000000000 P 03f5aa499bf14e27821fc840af155fcb: Log is configured to *not* fsync() on all Append() calls
I20260619 10:19:31.081230 14305 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 03f5aa499bf14e27821fc840af155fcb: No bootstrap required, opened a new log
Jun 19 10:19:31 dist-test-slave-6pt2 krb5kdc[14074](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864370, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.206.125@KRBTEST.COM
I20260619 10:19:31.082880 14305 raft_consensus.cc:348] T 00000000000000000000000000000000 P 03f5aa499bf14e27821fc840af155fcb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9de25348fd9f407b8bc3872ba204e794" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 44709 } } peers { permanent_uuid: "0a3241cf26ce49748ad65b39ae174525" member_type: VOTER last_known_addr { host: "127.0.206.125" port: 40605 } } peers { permanent_uuid: "03f5aa499bf14e27821fc840af155fcb" member_type: VOTER last_known_addr { host: "127.0.206.124" port: 39317 } }
I20260619 10:19:31.083015 14305 raft_consensus.cc:374] T 00000000000000000000000000000000 P 03f5aa499bf14e27821fc840af155fcb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 10:19:31.083067 14305 raft_consensus.cc:729] T 00000000000000000000000000000000 P 03f5aa499bf14e27821fc840af155fcb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 03f5aa499bf14e27821fc840af155fcb, State: Initialized, Role: FOLLOWER
I20260619 10:19:31.083202 14305 consensus_queue.cc:260] T 00000000000000000000000000000000 P 03f5aa499bf14e27821fc840af155fcb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9de25348fd9f407b8bc3872ba204e794" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 44709 } } peers { permanent_uuid: "0a3241cf26ce49748ad65b39ae174525" member_type: VOTER last_known_addr { host: "127.0.206.125" port: 40605 } } peers { permanent_uuid: "03f5aa499bf14e27821fc840af155fcb" member_type: VOTER last_known_addr { host: "127.0.206.124" port: 39317 } }
I20260619 10:19:31.083504 14305 sys_catalog.cc:564] T 00000000000000000000000000000000 P 03f5aa499bf14e27821fc840af155fcb [sys.catalog]: configured and running, proceeding with master startup.
I20260619 10:19:31.083477 14311 sys_catalog.cc:455] T 00000000000000000000000000000000 P 03f5aa499bf14e27821fc840af155fcb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9de25348fd9f407b8bc3872ba204e794" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 44709 } } peers { permanent_uuid: "0a3241cf26ce49748ad65b39ae174525" member_type: VOTER last_known_addr { host: "127.0.206.125" port: 40605 } } peers { permanent_uuid: "03f5aa499bf14e27821fc840af155fcb" member_type: VOTER last_known_addr { host: "127.0.206.124" port: 39317 } } }
I20260619 10:19:31.083667 14311 sys_catalog.cc:458] T 00000000000000000000000000000000 P 03f5aa499bf14e27821fc840af155fcb [sys.catalog]: This master's current role is: FOLLOWER
W20260619 10:19:31.085454 14322 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 03f5aa499bf14e27821fc840af155fcb: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 10:19:31.085521 14322 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 10:19:31.085882 14234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:19:31.077701 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37844 (local address 127.0.206.125:40605)
0619 10:19:31.077736 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0619 10:19:31.077738 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 10:19:31.077831 (+ 93us) server_negotiation.cc:407] Connection header received
0619 10:19:31.077878 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:19:31.077879 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:19:31.077890 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:19:31.077963 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:19:31.078497 (+ 534us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:19:31.079013 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:19:31.079959 (+ 946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:19:31.080158 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:19:31.080217 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:19:31.082461 (+ 2244us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 10:19:31.082463 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 10:19:31.082463 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 10:19:31.082470 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 10:19:31.084168 (+ 1698us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:19:31.084488 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:19:31.084490 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:19:31.084490 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:19:31.084550 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:19:31.084804 (+ 254us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:19:31.084805 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:19:31.084805 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:19:31.084941 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 10:19:31.085114 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0619 10:19:31.085719 (+ 605us) server_negotiation.cc:299] Negotiation successful
0619 10:19:31.085791 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260619 10:19:31.087617 825 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 10:19:31.085724 14237 master_runner.cc:186] Error getting master registration for 127.0.206.126:44709: OK, instance_id { permanent_uuid: "9de25348fd9f407b8bc3872ba204e794" instance_seqno: 1781864370749716 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260619 10:19:31.088203 14237 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260619 10:19:31.093757 14155 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.206.124" port: 39317 } attempt: 2
I20260619 10:19:31.097357 14159 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:19:31.088959 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48974 (local address 127.0.206.126:44709)
0619 10:19:31.089091 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0619 10:19:31.089093 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 10:19:31.089308 (+ 215us) server_negotiation.cc:407] Connection header received
0619 10:19:31.089354 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:19:31.089355 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:19:31.089365 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:19:31.089430 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:19:31.089906 (+ 476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:19:31.090653 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:19:31.091703 (+ 1050us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:19:31.091985 (+ 282us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:19:31.092023 (+ 38us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:19:31.093452 (+ 1429us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 10:19:31.093453 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 10:19:31.093454 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 10:19:31.093460 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 10:19:31.095838 (+ 2378us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:19:31.096439 (+ 601us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:19:31.096440 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:19:31.096441 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:19:31.096499 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:19:31.096725 (+ 226us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:19:31.096725 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:19:31.096726 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:19:31.096851 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 10:19:31.096926 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0619 10:19:31.097180 (+ 254us) server_negotiation.cc:299] Negotiation successful
0619 10:19:31.097256 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":110}
I20260619 10:19:31.098409 14234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:19:31.089196 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37858 (local address 127.0.206.125:40605)
0619 10:19:31.089236 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0619 10:19:31.089238 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 10:19:31.090896 (+ 1658us) server_negotiation.cc:407] Connection header received
0619 10:19:31.090984 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:19:31.090985 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:19:31.090994 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:19:31.091021 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:19:31.091686 (+ 665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:19:31.092337 (+ 651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:19:31.093380 (+ 1043us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:19:31.093676 (+ 296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:19:31.093716 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:19:31.095433 (+ 1717us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 10:19:31.095434 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 10:19:31.095435 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 10:19:31.095442 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 10:19:31.097111 (+ 1669us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:19:31.097516 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:19:31.097517 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:19:31.097517 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:19:31.097582 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:19:31.097860 (+ 278us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:19:31.097861 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:19:31.097862 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:19:31.097982 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 10:19:31.098048 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0619 10:19:31.098233 (+ 185us) server_negotiation.cc:299] Negotiation successful
0619 10:19:31.098305 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260619 10:19:31.098433 14309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:19:31.089038 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35186 (local address 127.0.206.124:39317)
0619 10:19:31.089093 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0619 10:19:31.089095 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 10:19:31.090756 (+ 1661us) server_negotiation.cc:407] Connection header received
0619 10:19:31.090820 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:19:31.090821 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:19:31.090831 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:19:31.090879 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:19:31.091527 (+ 648us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:19:31.092227 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:19:31.093202 (+ 975us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:19:31.093482 (+ 280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:19:31.093519 (+ 37us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:19:31.095036 (+ 1517us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 10:19:31.095037 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 10:19:31.095038 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 10:19:31.095044 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 10:19:31.096861 (+ 1817us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:19:31.097257 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:19:31.097258 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:19:31.097258 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:19:31.097310 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:19:31.097833 (+ 523us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:19:31.097834 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:19:31.097834 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:19:31.097987 (+ 153us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 10:19:31.098063 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0619 10:19:31.098267 (+ 204us) server_negotiation.cc:299] Negotiation successful
0619 10:19:31.098349 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
Jun 19 10:19:31 dist-test-slave-6pt2 krb5kdc[14074](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864370, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.126@KRBTEST.COM for kudu/127.0.206.124@KRBTEST.COM
I20260619 10:19:31.105129 14309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:19:31.094012 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35188 (local address 127.0.206.124:39317)
0619 10:19:31.098505 (+ 4493us) server_negotiation.cc:206] Beginning negotiation
0619 10:19:31.098507 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 10:19:31.098513 (+ 6us) server_negotiation.cc:407] Connection header received
0619 10:19:31.098529 (+ 16us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:19:31.098530 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:19:31.098539 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:19:31.098644 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:19:31.099355 (+ 711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:19:31.099879 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:19:31.100528 (+ 649us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:19:31.100735 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:19:31.100817 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:19:31.102690 (+ 1873us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 10:19:31.102691 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 10:19:31.102691 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 10:19:31.102697 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 10:19:31.104027 (+ 1330us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:19:31.104383 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:19:31.104384 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:19:31.104385 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:19:31.104434 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:19:31.104641 (+ 207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:19:31.104642 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:19:31.104643 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:19:31.104747 (+ 104us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 10:19:31.104816 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0619 10:19:31.104969 (+ 153us) server_negotiation.cc:299] Negotiation successful
0619 10:19:31.105034 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4482,"thread_start_us":81,"threads_started":1}
I20260619 10:19:31.105129 14158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:19:31.093967 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.124:39317 (local address 127.0.0.1:35188)
0619 10:19:31.094007 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0619 10:19:31.094018 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 10:19:31.094063 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 10:19:31.099111 (+ 5048us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 10:19:31.099112 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 10:19:31.099260 (+ 148us) client_negotiation.cc:189] Negotiated authn=SASL
0619 10:19:31.099298 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:19:31.099300 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:19:31.099967 (+ 667us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:19:31.099968 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:19:31.100448 (+ 480us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:19:31.100450 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:19:31.100819 (+ 369us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:19:31.100820 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:19:31.100871 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:19:31.101258 (+ 387us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 10:19:31.101262 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 10:19:31.102563 (+ 1301us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 10:19:31.104136 (+ 1573us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 10:19:31.104137 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 10:19:31.104138 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 10:19:31.104301 (+ 163us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 10:19:31.104509 (+ 208us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 10:19:31.104509 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 10:19:31.104510 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 10:19:31.104572 (+ 62us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 10:19:31.104821 (+ 249us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 10:19:31.104822 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 10:19:31.104862 (+ 40us) client_negotiation.cc:769] Sending connection context
0619 10:19:31.104945 (+ 83us) client_negotiation.cc:240] Negotiation successful
0619 10:19:31.105032 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":22}
I20260619 10:19:31.106329 14155 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9de25348fd9f407b8bc3872ba204e794: Bootstrap starting.
I20260619 10:19:31.106986 14155 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9de25348fd9f407b8bc3872ba204e794: Neither blocks nor log segments found. Creating new log.
I20260619 10:19:31.107272 14155 log.cc:824] T 00000000000000000000000000000000 P 9de25348fd9f407b8bc3872ba204e794: Log is configured to *not* fsync() on all Append() calls
I20260619 10:19:31.108026 14155 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9de25348fd9f407b8bc3872ba204e794: No bootstrap required, opened a new log
I20260619 10:19:31.109686 14155 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9de25348fd9f407b8bc3872ba204e794 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9de25348fd9f407b8bc3872ba204e794" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 44709 } } peers { permanent_uuid: "0a3241cf26ce49748ad65b39ae174525" member_type: VOTER last_known_addr { host: "127.0.206.125" port: 40605 } } peers { permanent_uuid: "03f5aa499bf14e27821fc840af155fcb" member_type: VOTER last_known_addr { host: "127.0.206.124" port: 39317 } }
I20260619 10:19:31.109789 14155 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9de25348fd9f407b8bc3872ba204e794 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 10:19:31.109810 14155 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9de25348fd9f407b8bc3872ba204e794 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9de25348fd9f407b8bc3872ba204e794, State: Initialized, Role: FOLLOWER
I20260619 10:19:31.109948 14155 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9de25348fd9f407b8bc3872ba204e794 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9de25348fd9f407b8bc3872ba204e794" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 44709 } } peers { permanent_uuid: "0a3241cf26ce49748ad65b39ae174525" member_type: VOTER last_known_addr { host: "127.0.206.125" port: 40605 } } peers { permanent_uuid: "03f5aa499bf14e27821fc840af155fcb" member_type: VOTER last_known_addr { host: "127.0.206.124" port: 39317 } }
I20260619 10:19:31.110237 14155 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9de25348fd9f407b8bc3872ba204e794 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 10:19:31.110219 14333 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9de25348fd9f407b8bc3872ba204e794 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9de25348fd9f407b8bc3872ba204e794" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 44709 } } peers { permanent_uuid: "0a3241cf26ce49748ad65b39ae174525" member_type: VOTER last_known_addr { host: "127.0.206.125" port: 40605 } } peers { permanent_uuid: "03f5aa499bf14e27821fc840af155fcb" member_type: VOTER last_known_addr { host: "127.0.206.124" port: 39317 } } }
I20260619 10:19:31.110335 14333 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9de25348fd9f407b8bc3872ba204e794 [sys.catalog]: This master's current role is: FOLLOWER
W20260619 10:19:31.111970 14344 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 9de25348fd9f407b8bc3872ba204e794: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 10:19:31.112069 14344 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 10:19:31.112520 14087 master_runner.cc:186] Error getting master registration for 127.0.206.125:40605: OK, instance_id { permanent_uuid: "0a3241cf26ce49748ad65b39ae174525" instance_seqno: 1781864370900909 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260619 10:19:31.115319 14087 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260619 10:19:31.136173 14230 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.206.124" port: 39317 } attempt: 3
I20260619 10:19:31.138952 14311 raft_consensus.cc:482] T 00000000000000000000000000000000 P 03f5aa499bf14e27821fc840af155fcb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 10:19:31.139056 14311 raft_consensus.cc:504] T 00000000000000000000000000000000 P 03f5aa499bf14e27821fc840af155fcb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9de25348fd9f407b8bc3872ba204e794" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 44709 } } peers { permanent_uuid: "0a3241cf26ce49748ad65b39ae174525" member_type: VOTER last_known_addr { host: "127.0.206.125" port: 40605 } } peers { permanent_uuid: "03f5aa499bf14e27821fc840af155fcb" member_type: VOTER last_known_addr { host: "127.0.206.124" port: 39317 } }
I20260619 10:19:31.139343 14311 leader_election.cc:288] T 00000000000000000000000000000000 P 03f5aa499bf14e27821fc840af155fcb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9de25348fd9f407b8bc3872ba204e794 (127.0.206.126:44709), 0a3241cf26ce49748ad65b39ae174525 (127.0.206.125:40605)
I20260619 10:19:31.140138 14129 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "03f5aa499bf14e27821fc840af155fcb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9de25348fd9f407b8bc3872ba204e794" is_pre_election: true
I20260619 10:19:31.140277 14129 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 9de25348fd9f407b8bc3872ba204e794 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 03f5aa499bf14e27821fc840af155fcb in term 0.
I20260619 10:19:31.140604 14256 leader_election.cc:302] T 00000000000000000000000000000000 P 03f5aa499bf14e27821fc840af155fcb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 03f5aa499bf14e27821fc840af155fcb, 9de25348fd9f407b8bc3872ba204e794; no voters:
Jun 19 10:19:31 dist-test-slave-6pt2 krb5kdc[14074](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864370, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.125@KRBTEST.COM for kudu/127.0.206.124@KRBTEST.COM
I20260619 10:19:31.141734 14311 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 03f5aa499bf14e27821fc840af155fcb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 10:19:31.141805 14311 raft_consensus.cc:482] T 00000000000000000000000000000000 P 03f5aa499bf14e27821fc840af155fcb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 10:19:31.141829 14311 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 03f5aa499bf14e27821fc840af155fcb [term 0 FOLLOWER]: Advancing to term 1
I20260619 10:19:31.142709 14311 raft_consensus.cc:504] T 00000000000000000000000000000000 P 03f5aa499bf14e27821fc840af155fcb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9de25348fd9f407b8bc3872ba204e794" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 44709 } } peers { permanent_uuid: "0a3241cf26ce49748ad65b39ae174525" member_type: VOTER last_known_addr { host: "127.0.206.125" port: 40605 } } peers { permanent_uuid: "03f5aa499bf14e27821fc840af155fcb" member_type: VOTER last_known_addr { host: "127.0.206.124" port: 39317 } }
I20260619 10:19:31.142817 14311 leader_election.cc:288] T 00000000000000000000000000000000 P 03f5aa499bf14e27821fc840af155fcb [CANDIDATE]: Term 1 election: Requested vote from peers 9de25348fd9f407b8bc3872ba204e794 (127.0.206.126:44709), 0a3241cf26ce49748ad65b39ae174525 (127.0.206.125:40605)
I20260619 10:19:31.143034 14129 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "03f5aa499bf14e27821fc840af155fcb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9de25348fd9f407b8bc3872ba204e794"
I20260619 10:19:31.143100 14129 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9de25348fd9f407b8bc3872ba204e794 [term 0 FOLLOWER]: Advancing to term 1
I20260619 10:19:31.144510 14129 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 9de25348fd9f407b8bc3872ba204e794 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 03f5aa499bf14e27821fc840af155fcb in term 1.
I20260619 10:19:31.144696 14256 leader_election.cc:302] T 00000000000000000000000000000000 P 03f5aa499bf14e27821fc840af155fcb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 03f5aa499bf14e27821fc840af155fcb, 9de25348fd9f407b8bc3872ba204e794; no voters:
I20260619 10:19:31.144781 14311 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 03f5aa499bf14e27821fc840af155fcb [term 1 FOLLOWER]: Leader election won for term 1
I20260619 10:19:31.144919 14311 raft_consensus.cc:686] T 00000000000000000000000000000000 P 03f5aa499bf14e27821fc840af155fcb [term 1 LEADER]: Becoming Leader. State: Replica: 03f5aa499bf14e27821fc840af155fcb, State: Running, Role: LEADER
I20260619 10:19:31.144997 14311 consensus_queue.cc:237] T 00000000000000000000000000000000 P 03f5aa499bf14e27821fc840af155fcb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9de25348fd9f407b8bc3872ba204e794" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 44709 } } peers { permanent_uuid: "0a3241cf26ce49748ad65b39ae174525" member_type: VOTER last_known_addr { host: "127.0.206.125" port: 40605 } } peers { permanent_uuid: "03f5aa499bf14e27821fc840af155fcb" member_type: VOTER last_known_addr { host: "127.0.206.124" port: 39317 } }
I20260619 10:19:31.145340 14311 sys_catalog.cc:455] T 00000000000000000000000000000000 P 03f5aa499bf14e27821fc840af155fcb [sys.catalog]: SysCatalogTable state changed. Reason: New leader 03f5aa499bf14e27821fc840af155fcb. Latest consensus state: current_term: 1 leader_uuid: "03f5aa499bf14e27821fc840af155fcb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9de25348fd9f407b8bc3872ba204e794" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 44709 } } peers { permanent_uuid: "0a3241cf26ce49748ad65b39ae174525" member_type: VOTER last_known_addr { host: "127.0.206.125" port: 40605 } } peers { permanent_uuid: "03f5aa499bf14e27821fc840af155fcb" member_type: VOTER last_known_addr { host: "127.0.206.124" port: 39317 } } }
I20260619 10:19:31.145401 14311 sys_catalog.cc:458] T 00000000000000000000000000000000 P 03f5aa499bf14e27821fc840af155fcb [sys.catalog]: This master's current role is: LEADER
I20260619 10:19:31.146693 14331 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:19:31.136661 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35192 (local address 127.0.206.124:39317)
0619 10:19:31.136738 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0619 10:19:31.136740 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 10:19:31.136748 (+ 8us) server_negotiation.cc:407] Connection header received
0619 10:19:31.136773 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:19:31.136773 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:19:31.136784 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:19:31.136860 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:19:31.137238 (+ 378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:19:31.138067 (+ 829us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:19:31.139108 (+ 1041us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:19:31.139437 (+ 329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:19:31.139495 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:19:31.143423 (+ 3928us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 10:19:31.143424 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 10:19:31.143425 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 10:19:31.143431 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 10:19:31.145168 (+ 1737us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:19:31.145633 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:19:31.145634 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:19:31.145635 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:19:31.145714 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:19:31.145986 (+ 272us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:19:31.145988 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:19:31.145988 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:19:31.146143 (+ 155us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 10:19:31.146187 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0619 10:19:31.146475 (+ 288us) server_negotiation.cc:299] Negotiation successful
0619 10:19:31.146569 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260619 10:19:31.147047 14232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:19:31.136450 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.124:39317 (local address 127.0.0.1:35192)
0619 10:19:31.136498 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0619 10:19:31.136519 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0619 10:19:31.136556 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 10:19:31.136897 (+ 341us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 10:19:31.136898 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 10:19:31.137099 (+ 201us) client_negotiation.cc:189] Negotiated authn=SASL
0619 10:19:31.137152 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:19:31.137154 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:19:31.138151 (+ 997us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:19:31.138152 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:19:31.139014 (+ 862us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:19:31.139016 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:19:31.139568 (+ 552us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:19:31.139569 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:19:31.139628 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:19:31.140130 (+ 502us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 10:19:31.140136 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0619 10:19:31.143321 (+ 3185us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 10:19:31.145324 (+ 2003us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 10:19:31.145325 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 10:19:31.145325 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 10:19:31.145539 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 10:19:31.145799 (+ 260us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 10:19:31.145800 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 10:19:31.145800 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 10:19:31.145887 (+ 87us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 10:19:31.146246 (+ 359us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 10:19:31.146247 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 10:19:31.146312 (+ 65us) client_negotiation.cc:769] Sending connection context
0619 10:19:31.146814 (+ 502us) client_negotiation.cc:240] Negotiation successful
0619 10:19:31.146949 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":30}
I20260619 10:19:31.145998 14348 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 10:19:31.147418 14348 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 10:19:31.148490 14230 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0a3241cf26ce49748ad65b39ae174525: Bootstrap starting.
I20260619 10:19:31.149181 14230 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0a3241cf26ce49748ad65b39ae174525: Neither blocks nor log segments found. Creating new log.
I20260619 10:19:31.149463 14230 log.cc:824] T 00000000000000000000000000000000 P 0a3241cf26ce49748ad65b39ae174525: Log is configured to *not* fsync() on all Append() calls
I20260619 10:19:31.150255 14230 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0a3241cf26ce49748ad65b39ae174525: No bootstrap required, opened a new log
I20260619 10:19:31.152151 14230 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0a3241cf26ce49748ad65b39ae174525 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9de25348fd9f407b8bc3872ba204e794" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 44709 } } peers { permanent_uuid: "0a3241cf26ce49748ad65b39ae174525" member_type: VOTER last_known_addr { host: "127.0.206.125" port: 40605 } } peers { permanent_uuid: "03f5aa499bf14e27821fc840af155fcb" member_type: VOTER last_known_addr { host: "127.0.206.124" port: 39317 } }
I20260619 10:19:31.152283 14230 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0a3241cf26ce49748ad65b39ae174525 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 10:19:31.152315 14230 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0a3241cf26ce49748ad65b39ae174525 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a3241cf26ce49748ad65b39ae174525, State: Initialized, Role: FOLLOWER
I20260619 10:19:31.152426 14230 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0a3241cf26ce49748ad65b39ae174525 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9de25348fd9f407b8bc3872ba204e794" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 44709 } } peers { permanent_uuid: "0a3241cf26ce49748ad65b39ae174525" member_type: VOTER last_known_addr { host: "127.0.206.125" port: 40605 } } peers { permanent_uuid: "03f5aa499bf14e27821fc840af155fcb" member_type: VOTER last_known_addr { host: "127.0.206.124" port: 39317 } }
I20260619 10:19:31.152720 14230 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0a3241cf26ce49748ad65b39ae174525 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 10:19:31.152999 14202 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0a3241cf26ce49748ad65b39ae174525 [term 0 FOLLOWER]: Advancing to term 1
I20260619 10:19:31.153838 14202 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 0a3241cf26ce49748ad65b39ae174525 [term 1 FOLLOWER]: Refusing update from remote peer 03f5aa499bf14e27821fc840af155fcb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 10:19:31.154219 14129 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 9de25348fd9f407b8bc3872ba204e794 [term 1 FOLLOWER]: Refusing update from remote peer 03f5aa499bf14e27821fc840af155fcb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 10:19:31.154444 14346 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 03f5aa499bf14e27821fc840af155fcb [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a3241cf26ce49748ad65b39ae174525" member_type: VOTER last_known_addr { host: "127.0.206.125" port: 40605 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 10:19:31.154214 14203 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "03f5aa499bf14e27821fc840af155fcb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a3241cf26ce49748ad65b39ae174525"
I20260619 10:19:31.155179 14311 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 03f5aa499bf14e27821fc840af155fcb [LEADER]: Connected to new peer: Peer: permanent_uuid: "9de25348fd9f407b8bc3872ba204e794" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 44709 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 10:19:31.154364 14204 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "03f5aa499bf14e27821fc840af155fcb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a3241cf26ce49748ad65b39ae174525" is_pre_election: true
I20260619 10:19:31.155769 14203 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 0a3241cf26ce49748ad65b39ae174525 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 03f5aa499bf14e27821fc840af155fcb in term 1.
I20260619 10:19:31.156033 14351 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0a3241cf26ce49748ad65b39ae174525 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9de25348fd9f407b8bc3872ba204e794" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 44709 } } peers { permanent_uuid: "0a3241cf26ce49748ad65b39ae174525" member_type: VOTER last_known_addr { host: "127.0.206.125" port: 40605 } } peers { permanent_uuid: "03f5aa499bf14e27821fc840af155fcb" member_type: VOTER last_known_addr { host: "127.0.206.124" port: 39317 } } }
I20260619 10:19:31.156111 14351 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0a3241cf26ce49748ad65b39ae174525 [sys.catalog]: This master's current role is: FOLLOWER
I20260619 10:19:31.157141 14333 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9de25348fd9f407b8bc3872ba204e794 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 03f5aa499bf14e27821fc840af155fcb. Latest consensus state: current_term: 1 leader_uuid: "03f5aa499bf14e27821fc840af155fcb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9de25348fd9f407b8bc3872ba204e794" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 44709 } } peers { permanent_uuid: "0a3241cf26ce49748ad65b39ae174525" member_type: VOTER last_known_addr { host: "127.0.206.125" port: 40605 } } peers { permanent_uuid: "03f5aa499bf14e27821fc840af155fcb" member_type: VOTER last_known_addr { host: "127.0.206.124" port: 39317 } } }
I20260619 10:19:31.157220 14333 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9de25348fd9f407b8bc3872ba204e794 [sys.catalog]: This master's current role is: FOLLOWER
I20260619 10:19:31.157879 14351 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0a3241cf26ce49748ad65b39ae174525 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 03f5aa499bf14e27821fc840af155fcb. Latest consensus state: current_term: 1 leader_uuid: "03f5aa499bf14e27821fc840af155fcb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9de25348fd9f407b8bc3872ba204e794" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 44709 } } peers { permanent_uuid: "0a3241cf26ce49748ad65b39ae174525" member_type: VOTER last_known_addr { host: "127.0.206.125" port: 40605 } } peers { permanent_uuid: "03f5aa499bf14e27821fc840af155fcb" member_type: VOTER last_known_addr { host: "127.0.206.124" port: 39317 } } }
I20260619 10:19:31.157958 14351 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0a3241cf26ce49748ad65b39ae174525 [sys.catalog]: This master's current role is: FOLLOWER
W20260619 10:19:31.158390 14362 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 0a3241cf26ce49748ad65b39ae174525: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 10:19:31.158435 14362 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 10:19:31.159471 14346 sys_catalog.cc:455] T 00000000000000000000000000000000 P 03f5aa499bf14e27821fc840af155fcb [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "03f5aa499bf14e27821fc840af155fcb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9de25348fd9f407b8bc3872ba204e794" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 44709 } } peers { permanent_uuid: "0a3241cf26ce49748ad65b39ae174525" member_type: VOTER last_known_addr { host: "127.0.206.125" port: 40605 } } peers { permanent_uuid: "03f5aa499bf14e27821fc840af155fcb" member_type: VOTER last_known_addr { host: "127.0.206.124" port: 39317 } } }
I20260619 10:19:31.159561 14346 sys_catalog.cc:458] T 00000000000000000000000000000000 P 03f5aa499bf14e27821fc840af155fcb [sys.catalog]: This master's current role is: LEADER
I20260619 10:19:31.160164 14351 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0a3241cf26ce49748ad65b39ae174525 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "03f5aa499bf14e27821fc840af155fcb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9de25348fd9f407b8bc3872ba204e794" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 44709 } } peers { permanent_uuid: "0a3241cf26ce49748ad65b39ae174525" member_type: VOTER last_known_addr { host: "127.0.206.125" port: 40605 } } peers { permanent_uuid: "03f5aa499bf14e27821fc840af155fcb" member_type: VOTER last_known_addr { host: "127.0.206.124" port: 39317 } } }
I20260619 10:19:31.160295 14351 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0a3241cf26ce49748ad65b39ae174525 [sys.catalog]: This master's current role is: FOLLOWER
I20260619 10:19:31.160395 14349 mvcc.cc:205] Tried to move back new op lower bound from 7298516464231534592 to 7298516464210509824. Current Snapshot: MvccSnapshot[applied={T|T < 7298516464231534592}]
I20260619 10:19:31.160529 14365 mvcc.cc:205] Tried to move back new op lower bound from 7298516464231534592 to 7298516464210509824. Current Snapshot: MvccSnapshot[applied={T|T < 7298516464231534592}]
I20260619 10:19:31.166782 14348 catalog_manager.cc:1347] Generated new cluster ID: a374b46ffa1c41dd9abf5ad1950e43d7
I20260619 10:19:31.166838 14348 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 10:19:31.167568 14311 sys_catalog.cc:455] T 00000000000000000000000000000000 P 03f5aa499bf14e27821fc840af155fcb [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "03f5aa499bf14e27821fc840af155fcb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9de25348fd9f407b8bc3872ba204e794" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 44709 } } peers { permanent_uuid: "0a3241cf26ce49748ad65b39ae174525" member_type: VOTER last_known_addr { host: "127.0.206.125" port: 40605 } } peers { permanent_uuid: "03f5aa499bf14e27821fc840af155fcb" member_type: VOTER last_known_addr { host: "127.0.206.124" port: 39317 } } }
I20260619 10:19:31.167649 14311 sys_catalog.cc:458] T 00000000000000000000000000000000 P 03f5aa499bf14e27821fc840af155fcb [sys.catalog]: This master's current role is: LEADER
I20260619 10:19:31.168046 14333 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9de25348fd9f407b8bc3872ba204e794 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "03f5aa499bf14e27821fc840af155fcb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9de25348fd9f407b8bc3872ba204e794" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 44709 } } peers { permanent_uuid: "0a3241cf26ce49748ad65b39ae174525" member_type: VOTER last_known_addr { host: "127.0.206.125" port: 40605 } } peers { permanent_uuid: "03f5aa499bf14e27821fc840af155fcb" member_type: VOTER last_known_addr { host: "127.0.206.124" port: 39317 } } }
I20260619 10:19:31.168130 14333 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9de25348fd9f407b8bc3872ba204e794 [sys.catalog]: This master's current role is: FOLLOWER
I20260619 10:19:31.168318 14363 mvcc.cc:205] Tried to move back new op lower bound from 7298516464231534592 to 7298516464210509824. Current Snapshot: MvccSnapshot[applied={T|T < 7298516464231534592}]
I20260619 10:19:31.180137 14348 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 10:19:31.180893 14348 catalog_manager.cc:1504] Loading token signing keys...
I20260619 10:19:31.186305 14348 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 03f5aa499bf14e27821fc840af155fcb: Generated new TSK 0
I20260619 10:19:31.186539 14348 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 10:19:32.112509 14344 catalog_manager.cc:1259] Loaded cluster ID: a374b46ffa1c41dd9abf5ad1950e43d7
I20260619 10:19:32.112576 14344 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 9de25348fd9f407b8bc3872ba204e794: loading cluster ID for follower catalog manager: success
I20260619 10:19:32.113318 14344 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 9de25348fd9f407b8bc3872ba204e794: acquiring CA information for follower catalog manager: success
I20260619 10:19:32.113552 14344 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 9de25348fd9f407b8bc3872ba204e794: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260619 10:19:32.158854 14362 catalog_manager.cc:1259] Loaded cluster ID: a374b46ffa1c41dd9abf5ad1950e43d7
I20260619 10:19:32.158931 14362 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 0a3241cf26ce49748ad65b39ae174525: loading cluster ID for follower catalog manager: success
I20260619 10:19:32.159683 14362 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 0a3241cf26ce49748ad65b39ae174525: acquiring CA information for follower catalog manager: success
I20260619 10:19:32.159926 14362 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 0a3241cf26ce49748ad65b39ae174525: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260619 10:19:32.239985 825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTU9OP6/build/release/bin/kudu with pid 14087
I20260619 10:19:32.245064 825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTU9OP6/build/release/bin/kudu with pid 14162
I20260619 10:19:32.249763 825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTU9OP6/build/release/bin/kudu with pid 14237
2026-06-19T10:19:32Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (2753 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781864224577058-825-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 10:19:32 dist-test-slave-6pt2 krb5kdc[14378](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 10:19:32 dist-test-slave-6pt2 krb5kdc[14378](info): set up 2 sockets
Jun 19 10:19:32 dist-test-slave-6pt2 krb5kdc[14378](info): commencing operation
krb5kdc: starting...
W20260619 10:19:33.286819 825 mini_kdc.cc:121] Time spent starting KDC: real 1.017s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781864224577058-825-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781864224577058-825-0/krb5kdc/test-user.keytab.
Jun 19 10:19:33 dist-test-slave-6pt2 krb5kdc[14378](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864373, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T10:19:33Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T10:19:33Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.206.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.206.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.206.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781864224577058-825-0/krb5kdc/kudu_127.0.206.126.keytab.
Entry for principal kudu/127.0.206.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781864224577058-825-0/krb5kdc/kudu_127.0.206.126.keytab.
I20260619 10:19:33.397830 825 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTU9OP6/build/release/bin/kudu
/tmp/dist-test-taskTU9OP6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781864224577058-825-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781864224577058-825-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781864224577058-825-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781864224577058-825-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.206.126:41003
--webserver_interface=127.0.206.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781864224577058-825-0/krb5kdc/kudu_127.0.206.126.keytab
--principal=kudu/127.0.206.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.206.84:37371
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.206.126:41003
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781864224577058-825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781864224577058-825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781864224577058-825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 10:19:33.477429 14391 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 10:19:33.477674 14391 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 10:19:33.477710 14391 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 10:19:33.478861 14391 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 10:19:33.478940 14391 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 10:19:33.478961 14391 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 10:19:33.478983 14391 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 10:19:33.479013 14391 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 10:19:33.480496 14391 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.206.84:37371
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781864224577058-825-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781864224577058-825-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.206.126:41003
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781864224577058-825-0/krb5kdc/kudu_127.0.206.126.keytab
--principal=kudu/127.0.206.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.206.126:41003
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781864224577058-825-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.206.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14391
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781864224577058-825-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 09:32:44 UTC on ccycloud.quasar-gbtldp.root.comops.site
build id 4455440
I20260619 10:19:33.480794 14391 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 10:19:33.481024 14391 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 10:19:33.485889 14399 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 10:19:33.485857 14397 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 10:19:33.485863 14396 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 10:19:33.485936 14391 server_base.cc:1034] running on GCE node
I20260619 10:19:33.486318 14391 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 10:19:33.486670 14391 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 10:19:33.487821 14391 hybrid_clock.cc:648] HybridClock initialized: now 1781864373487811 us; error 28 us; skew 500 ppm
Jun 19 10:19:33 dist-test-slave-6pt2 krb5kdc[14378](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864373, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 10:19:33.490355 14391 init.cc:377] Logged in from keytab as kudu/127.0.206.126@KRBTEST.COM (short username kudu)
I20260619 10:19:33.490861 14391 webserver.cc:460] Webserver started at http://127.0.206.126:43617/ using document root <none> and password file <none>
I20260619 10:19:33.491058 14391 fs_manager.cc:359] Metadata directory not provided
I20260619 10:19:33.491099 14391 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 10:19:33.491194 14391 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 10:19:33.492226 14391 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781864224577058-825-0/minicluster-data/master-0/data/instance:
uuid: "5238264c9a6444d18e7af6007f2839f7"
format_stamp: "Formatted at 2026-06-19 10:19:33 on dist-test-slave-6pt2"
server_key: "3f4773ac756fe355edb5b9e266f2fe2c"
server_key_iv: "8bc088e5765e9cc63f1aedec82882c17"
server_key_version: "encryptionkey@0"
I20260619 10:19:33.492527 14391 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781864224577058-825-0/minicluster-data/master-0/wal/instance:
uuid: "5238264c9a6444d18e7af6007f2839f7"
format_stamp: "Formatted at 2026-06-19 10:19:33 on dist-test-slave-6pt2"
server_key: "3f4773ac756fe355edb5b9e266f2fe2c"
server_key_iv: "8bc088e5765e9cc63f1aedec82882c17"
server_key_version: "encryptionkey@0"
I20260619 10:19:33.493752 14391 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260619 10:19:33.494374 14406 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 10:19:33.494546 14391 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260619 10:19:33.494683 14391 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781864224577058-825-0/minicluster-data/master-0/data,/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781864224577058-825-0/minicluster-data/master-0/wal
uuid: "5238264c9a6444d18e7af6007f2839f7"
format_stamp: "Formatted at 2026-06-19 10:19:33 on dist-test-slave-6pt2"
server_key: "3f4773ac756fe355edb5b9e266f2fe2c"
server_key_iv: "8bc088e5765e9cc63f1aedec82882c17"
server_key_version: "encryptionkey@0"
I20260619 10:19:33.494742 14391 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781864224577058-825-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781864224577058-825-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781864224577058-825-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 10:19:33.502391 14391 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 10:19:33.504153 14391 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 10:19:33.504259 14391 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 10:19:33.507975 14391 rpc_server.cc:307] RPC server started. Bound to: 127.0.206.126:41003
I20260619 10:19:33.508025 14458 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.206.126:41003 every 8 connection(s)
I20260619 10:19:33.508345 14391 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781864224577058-825-0/minicluster-data/master-0/data/info.pb
I20260619 10:19:33.509210 14459 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 10:19:33.511508 14459 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5238264c9a6444d18e7af6007f2839f7: Bootstrap starting.
I20260619 10:19:33.512108 14459 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5238264c9a6444d18e7af6007f2839f7: Neither blocks nor log segments found. Creating new log.
I20260619 10:19:33.512118 825 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTU9OP6/build/release/bin/kudu as pid 14391
I20260619 10:19:33.512193 825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781864224577058-825-0/minicluster-data/master-0/wal/instance
I20260619 10:19:33.512403 825 external_mini_cluster.cc:1428] Setting key 156d59865f45c97fc79f93c84cd8d406
I20260619 10:19:33.512430 14459 log.cc:824] T 00000000000000000000000000000000 P 5238264c9a6444d18e7af6007f2839f7: Log is configured to *not* fsync() on all Append() calls
I20260619 10:19:33.513259 14459 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5238264c9a6444d18e7af6007f2839f7: No bootstrap required, opened a new log
I20260619 10:19:33.515100 14459 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5238264c9a6444d18e7af6007f2839f7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5238264c9a6444d18e7af6007f2839f7" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 41003 } }
I20260619 10:19:33.515275 14459 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5238264c9a6444d18e7af6007f2839f7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 10:19:33.515311 14459 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5238264c9a6444d18e7af6007f2839f7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5238264c9a6444d18e7af6007f2839f7, State: Initialized, Role: FOLLOWER
I20260619 10:19:33.515436 14459 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5238264c9a6444d18e7af6007f2839f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5238264c9a6444d18e7af6007f2839f7" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 41003 } }
I20260619 10:19:33.515491 14459 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5238264c9a6444d18e7af6007f2839f7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 10:19:33.515511 14459 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5238264c9a6444d18e7af6007f2839f7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 10:19:33.515538 14459 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5238264c9a6444d18e7af6007f2839f7 [term 0 FOLLOWER]: Advancing to term 1
I20260619 10:19:33.516474 14459 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5238264c9a6444d18e7af6007f2839f7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5238264c9a6444d18e7af6007f2839f7" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 41003 } }
I20260619 10:19:33.516583 14459 leader_election.cc:302] T 00000000000000000000000000000000 P 5238264c9a6444d18e7af6007f2839f7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5238264c9a6444d18e7af6007f2839f7; no voters:
I20260619 10:19:33.516738 14459 leader_election.cc:288] T 00000000000000000000000000000000 P 5238264c9a6444d18e7af6007f2839f7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 10:19:33.516800 14464 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5238264c9a6444d18e7af6007f2839f7 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 10:19:33.516966 14464 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5238264c9a6444d18e7af6007f2839f7 [term 1 LEADER]: Becoming Leader. State: Replica: 5238264c9a6444d18e7af6007f2839f7, State: Running, Role: LEADER
I20260619 10:19:33.516985 14459 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5238264c9a6444d18e7af6007f2839f7 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 10:19:33.518359 14464 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5238264c9a6444d18e7af6007f2839f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5238264c9a6444d18e7af6007f2839f7" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 41003 } }
I20260619 10:19:33.518929 14465 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5238264c9a6444d18e7af6007f2839f7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5238264c9a6444d18e7af6007f2839f7. Latest consensus state: current_term: 1 leader_uuid: "5238264c9a6444d18e7af6007f2839f7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5238264c9a6444d18e7af6007f2839f7" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 41003 } } }
I20260619 10:19:33.519090 14465 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5238264c9a6444d18e7af6007f2839f7 [sys.catalog]: This master's current role is: LEADER
I20260619 10:19:33.519110 14466 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5238264c9a6444d18e7af6007f2839f7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5238264c9a6444d18e7af6007f2839f7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5238264c9a6444d18e7af6007f2839f7" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 41003 } } }
I20260619 10:19:33.519177 14466 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5238264c9a6444d18e7af6007f2839f7 [sys.catalog]: This master's current role is: LEADER
W20260619 10:19:33.519704 14478 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 5238264c9a6444d18e7af6007f2839f7: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 10:19:33.519760 14478 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 10:19:33.519856 14479 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 10:19:33.520012 14479 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Jun 19 10:19:33 dist-test-slave-6pt2 krb5kdc[14378](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864373, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.206.126@KRBTEST.COM
I20260619 10:19:33.522384 14479 catalog_manager.cc:1347] Generated new cluster ID: db7b7e2646bb428da9865abcb8c841d1
I20260619 10:19:33.522480 14479 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 10:19:33.525738 14463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:19:33.512907 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38906 (local address 127.0.206.126:41003)
0619 10:19:33.513087 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0619 10:19:33.513097 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0619 10:19:33.514858 (+ 1761us) server_negotiation.cc:407] Connection header received
0619 10:19:33.515058 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:19:33.515066 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:19:33.515125 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:19:33.515278 (+ 153us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:19:33.515958 (+ 680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:19:33.517377 (+ 1419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:19:33.518538 (+ 1161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:19:33.519246 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:19:33.519352 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:19:33.521979 (+ 2627us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 10:19:33.521983 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 10:19:33.521984 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 10:19:33.521996 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0619 10:19:33.524017 (+ 2021us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:19:33.524397 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:19:33.524400 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:19:33.524401 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:19:33.524534 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:19:33.524736 (+ 202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:19:33.524737 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:19:33.524738 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:19:33.525047 (+ 309us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 10:19:33.525260 (+ 213us) server_negotiation.cc:1035] Waiting for connection context
0619 10:19:33.525347 (+ 87us) server_negotiation.cc:299] Negotiation successful
0619 10:19:33.525520 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":128,"thread_start_us":79,"threads_started":1}
I20260619 10:19:33.526731 825 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 10:19:33.541260 14479 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 10:19:33.541810 14479 catalog_manager.cc:1504] Loading token signing keys...
I20260619 10:19:33.545828 14479 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5238264c9a6444d18e7af6007f2839f7: Generated new TSK 0
I20260619 10:19:33.546010 14479 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 10:19:33.580523 825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTU9OP6/build/release/bin/kudu with pid 14391
2026-06-19T10:19:33Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (1329 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781864224577058-825-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 10:19:33 dist-test-slave-6pt2 krb5kdc[14486](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 10:19:33 dist-test-slave-6pt2 krb5kdc[14486](info): set up 2 sockets
Jun 19 10:19:33 dist-test-slave-6pt2 krb5kdc[14486](info): commencing operation
krb5kdc: starting...
W20260619 10:19:34.620368 825 mini_kdc.cc:121] Time spent starting KDC: real 1.022s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781864224577058-825-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781864224577058-825-0/krb5kdc/test-user.keytab.
Jun 19 10:19:34 dist-test-slave-6pt2 krb5kdc[14486](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864374, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T10:19:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T10:19:34Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.206.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.206.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.206.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781864224577058-825-0/krb5kdc/kudu_127.0.206.126.keytab.
Entry for principal kudu/127.0.206.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781864224577058-825-0/krb5kdc/kudu_127.0.206.126.keytab.
I20260619 10:19:34.734476 825 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTU9OP6/build/release/bin/kudu
/tmp/dist-test-taskTU9OP6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781864224577058-825-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781864224577058-825-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781864224577058-825-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781864224577058-825-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.206.126:36823
--webserver_interface=127.0.206.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781864224577058-825-0/krb5kdc/kudu_127.0.206.126.keytab
--principal=kudu/127.0.206.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.206.84:42497
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.206.126:36823
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781864224577058-825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781864224577058-825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781864224577058-825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 10:19:34.812171 14499 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 10:19:34.812359 14499 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 10:19:34.812381 14499 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 10:19:34.813491 14499 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 10:19:34.813531 14499 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 10:19:34.813545 14499 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 10:19:34.813558 14499 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 10:19:34.813616 14499 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 10:19:34.815025 14499 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.206.84:42497
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781864224577058-825-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781864224577058-825-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.206.126:36823
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781864224577058-825-0/krb5kdc/kudu_127.0.206.126.keytab
--principal=kudu/127.0.206.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.206.126:36823
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781864224577058-825-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.206.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14499
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781864224577058-825-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 09:32:44 UTC on ccycloud.quasar-gbtldp.root.comops.site
build id 4455440
I20260619 10:19:34.815227 14499 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 10:19:34.815418 14499 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 10:19:34.820086 14504 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 10:19:34.820086 14507 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 10:19:34.820252 14499 server_base.cc:1034] running on GCE node
W20260619 10:19:34.820156 14505 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 10:19:34.820516 14499 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 10:19:34.820794 14499 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 10:19:34.821955 14499 hybrid_clock.cc:648] HybridClock initialized: now 1781864374821934 us; error 34 us; skew 500 ppm
Jun 19 10:19:34 dist-test-slave-6pt2 krb5kdc[14486](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864374, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 10:19:34.824294 14499 init.cc:377] Logged in from keytab as kudu/127.0.206.126@KRBTEST.COM (short username kudu)
I20260619 10:19:34.824733 14499 webserver.cc:460] Webserver started at http://127.0.206.126:36799/ using document root <none> and password file <none>
I20260619 10:19:34.824947 14499 fs_manager.cc:359] Metadata directory not provided
I20260619 10:19:34.825013 14499 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 10:19:34.825163 14499 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 10:19:34.826152 14499 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781864224577058-825-0/minicluster-data/master-0/data/instance:
uuid: "33de2c9a9d4148a281c87656d617ca65"
format_stamp: "Formatted at 2026-06-19 10:19:34 on dist-test-slave-6pt2"
server_key: "df2feca8e810e6795a30dfc48cdad65a"
server_key_iv: "fdff8c532fcc333feada82355159374c"
server_key_version: "encryptionkey@0"
I20260619 10:19:34.826493 14499 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781864224577058-825-0/minicluster-data/master-0/wal/instance:
uuid: "33de2c9a9d4148a281c87656d617ca65"
format_stamp: "Formatted at 2026-06-19 10:19:34 on dist-test-slave-6pt2"
server_key: "df2feca8e810e6795a30dfc48cdad65a"
server_key_iv: "fdff8c532fcc333feada82355159374c"
server_key_version: "encryptionkey@0"
I20260619 10:19:34.827764 14499 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260619 10:19:34.828450 14514 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 10:19:34.828613 14499 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260619 10:19:34.828707 14499 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781864224577058-825-0/minicluster-data/master-0/data,/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781864224577058-825-0/minicluster-data/master-0/wal
uuid: "33de2c9a9d4148a281c87656d617ca65"
format_stamp: "Formatted at 2026-06-19 10:19:34 on dist-test-slave-6pt2"
server_key: "df2feca8e810e6795a30dfc48cdad65a"
server_key_iv: "fdff8c532fcc333feada82355159374c"
server_key_version: "encryptionkey@0"
I20260619 10:19:34.828787 14499 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781864224577058-825-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781864224577058-825-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781864224577058-825-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 10:19:34.842319 14499 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 10:19:34.843986 14499 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 10:19:34.844141 14499 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 10:19:34.848002 14499 rpc_server.cc:307] RPC server started. Bound to: 127.0.206.126:36823
I20260619 10:19:34.848032 14566 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.206.126:36823 every 8 connection(s)
I20260619 10:19:34.848349 14499 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781864224577058-825-0/minicluster-data/master-0/data/info.pb
I20260619 10:19:34.849244 14567 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 10:19:34.851400 14567 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 33de2c9a9d4148a281c87656d617ca65: Bootstrap starting.
I20260619 10:19:34.851894 14567 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 33de2c9a9d4148a281c87656d617ca65: Neither blocks nor log segments found. Creating new log.
I20260619 10:19:34.852119 14567 log.cc:824] T 00000000000000000000000000000000 P 33de2c9a9d4148a281c87656d617ca65: Log is configured to *not* fsync() on all Append() calls
I20260619 10:19:34.852717 14567 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 33de2c9a9d4148a281c87656d617ca65: No bootstrap required, opened a new log
I20260619 10:19:34.854182 14567 raft_consensus.cc:348] T 00000000000000000000000000000000 P 33de2c9a9d4148a281c87656d617ca65 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "33de2c9a9d4148a281c87656d617ca65" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 36823 } }
I20260619 10:19:34.854321 14567 raft_consensus.cc:374] T 00000000000000000000000000000000 P 33de2c9a9d4148a281c87656d617ca65 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 10:19:34.854343 14567 raft_consensus.cc:729] T 00000000000000000000000000000000 P 33de2c9a9d4148a281c87656d617ca65 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 33de2c9a9d4148a281c87656d617ca65, State: Initialized, Role: FOLLOWER
I20260619 10:19:34.854472 14567 consensus_queue.cc:260] T 00000000000000000000000000000000 P 33de2c9a9d4148a281c87656d617ca65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "33de2c9a9d4148a281c87656d617ca65" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 36823 } }
I20260619 10:19:34.854554 14567 raft_consensus.cc:388] T 00000000000000000000000000000000 P 33de2c9a9d4148a281c87656d617ca65 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 10:19:34.854578 14567 raft_consensus.cc:482] T 00000000000000000000000000000000 P 33de2c9a9d4148a281c87656d617ca65 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 10:19:34.854648 14567 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 33de2c9a9d4148a281c87656d617ca65 [term 0 FOLLOWER]: Advancing to term 1
I20260619 10:19:34.855392 14567 raft_consensus.cc:504] T 00000000000000000000000000000000 P 33de2c9a9d4148a281c87656d617ca65 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "33de2c9a9d4148a281c87656d617ca65" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 36823 } }
I20260619 10:19:34.855474 14567 leader_election.cc:302] T 00000000000000000000000000000000 P 33de2c9a9d4148a281c87656d617ca65 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 33de2c9a9d4148a281c87656d617ca65; no voters:
I20260619 10:19:34.855585 14567 leader_election.cc:288] T 00000000000000000000000000000000 P 33de2c9a9d4148a281c87656d617ca65 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 10:19:34.855654 14570 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 33de2c9a9d4148a281c87656d617ca65 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 10:19:34.855818 14570 raft_consensus.cc:686] T 00000000000000000000000000000000 P 33de2c9a9d4148a281c87656d617ca65 [term 1 LEADER]: Becoming Leader. State: Replica: 33de2c9a9d4148a281c87656d617ca65, State: Running, Role: LEADER
I20260619 10:19:34.855829 14567 sys_catalog.cc:564] T 00000000000000000000000000000000 P 33de2c9a9d4148a281c87656d617ca65 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 10:19:34.855914 14570 consensus_queue.cc:237] T 00000000000000000000000000000000 P 33de2c9a9d4148a281c87656d617ca65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "33de2c9a9d4148a281c87656d617ca65" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 36823 } }
I20260619 10:19:34.856293 14572 sys_catalog.cc:455] T 00000000000000000000000000000000 P 33de2c9a9d4148a281c87656d617ca65 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 33de2c9a9d4148a281c87656d617ca65. Latest consensus state: current_term: 1 leader_uuid: "33de2c9a9d4148a281c87656d617ca65" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "33de2c9a9d4148a281c87656d617ca65" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 36823 } } }
I20260619 10:19:34.856285 14571 sys_catalog.cc:455] T 00000000000000000000000000000000 P 33de2c9a9d4148a281c87656d617ca65 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "33de2c9a9d4148a281c87656d617ca65" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "33de2c9a9d4148a281c87656d617ca65" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 36823 } } }
I20260619 10:19:34.856388 14572 sys_catalog.cc:458] T 00000000000000000000000000000000 P 33de2c9a9d4148a281c87656d617ca65 [sys.catalog]: This master's current role is: LEADER
I20260619 10:19:34.856392 14571 sys_catalog.cc:458] T 00000000000000000000000000000000 P 33de2c9a9d4148a281c87656d617ca65 [sys.catalog]: This master's current role is: LEADER
I20260619 10:19:34.856884 14580 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 10:19:34.857396 14580 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 10:19:34.858500 825 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTU9OP6/build/release/bin/kudu as pid 14499
I20260619 10:19:34.858623 825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781864224577058-825-0/minicluster-data/master-0/wal/instance
I20260619 10:19:34.858850 825 external_mini_cluster.cc:1428] Setting key f505c682c23acc53701af5eea6f0fc70
I20260619 10:19:34.859397 14580 catalog_manager.cc:1347] Generated new cluster ID: ffbf713c1b564873b61b865becdfe3dd
I20260619 10:19:34.859445 14580 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Jun 19 10:19:34 dist-test-slave-6pt2 krb5kdc[14486](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864374, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.206.126@KRBTEST.COM
I20260619 10:19:34.873350 14580 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 10:19:34.873435 14590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:19:34.859268 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49936 (local address 127.0.206.126:36823)
0619 10:19:34.862711 (+ 3443us) server_negotiation.cc:206] Beginning negotiation
0619 10:19:34.862720 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 10:19:34.862751 (+ 31us) server_negotiation.cc:407] Connection header received
0619 10:19:34.862852 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:19:34.862859 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:19:34.862908 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:19:34.863042 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:19:34.863808 (+ 766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:19:34.865187 (+ 1379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:19:34.866282 (+ 1095us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:19:34.866879 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:19:34.866980 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:19:34.869863 (+ 2883us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 10:19:34.869867 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 10:19:34.869868 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 10:19:34.869878 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0619 10:19:34.871798 (+ 1920us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:19:34.872185 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:19:34.872187 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:19:34.872188 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:19:34.872284 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:19:34.872499 (+ 215us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:19:34.872500 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:19:34.872500 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:19:34.872772 (+ 272us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 10:19:34.872916 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0619 10:19:34.873042 (+ 126us) server_negotiation.cc:299] Negotiation successful
0619 10:19:34.873184 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3401,"thread_start_us":72,"threads_started":1}
I20260619 10:19:34.874042 14580 catalog_manager.cc:1504] Loading token signing keys...
I20260619 10:19:34.879060 14580 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 33de2c9a9d4148a281c87656d617ca65: Generated new TSK 0
I20260619 10:19:34.879237 14580 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.0.206.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.206.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.206.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781864224577058-825-0/krb5kdc/kudu_127.0.206.65.keytab.
Entry for principal kudu/127.0.206.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781864224577058-825-0/krb5kdc/kudu_127.0.206.65.keytab.
I20260619 10:19:34.907192 825 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTU9OP6/build/release/bin/kudu
/tmp/dist-test-taskTU9OP6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781864224577058-825-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781864224577058-825-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781864224577058-825-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781864224577058-825-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.206.65:0
--local_ip_for_outbound_sockets=127.0.206.65
--webserver_interface=127.0.206.65
--webserver_port=0
--tserver_master_addrs=127.0.206.126:36823
--keytab_file=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781864224577058-825-0/krb5kdc/kudu_127.0.206.65.keytab
--principal=kudu/127.0.206.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.206.84:42497
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781864224577058-825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781864224577058-825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781864224577058-825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 10:19:35.001139 14593 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 10:19:35.001325 14593 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 10:19:35.001348 14593 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 10:19:35.002447 14593 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 10:19:35.002492 14593 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 10:19:35.002518 14593 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.206.65
I20260619 10:19:35.003924 14593 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.206.84:42497
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781864224577058-825-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781864224577058-825-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781864224577058-825-0/krb5kdc/kudu_127.0.206.65.keytab
--principal=kudu/127.0.206.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.206.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781864224577058-825-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.206.65
--webserver_port=0
--tserver_master_addrs=127.0.206.126:36823
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14593
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.206.65
--log_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781864224577058-825-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 09:32:44 UTC on ccycloud.quasar-gbtldp.root.comops.site
build id 4455440
I20260619 10:19:35.004138 14593 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 10:19:35.004384 14593 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 10:19:35.009117 14601 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 10:19:35.009130 14599 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 10:19:35.009305 14598 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 10:19:35.009312 14593 server_base.cc:1034] running on GCE node
I20260619 10:19:35.009618 14593 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 10:19:35.009881 14593 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 10:19:35.011056 14593 hybrid_clock.cc:648] HybridClock initialized: now 1781864375011041 us; error 32 us; skew 500 ppm
Jun 19 10:19:35 dist-test-slave-6pt2 krb5kdc[14486](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864375, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 10:19:35.013595 14593 init.cc:377] Logged in from keytab as kudu/127.0.206.65@KRBTEST.COM (short username kudu)
I20260619 10:19:35.014006 14593 webserver.cc:460] Webserver started at http://127.0.206.65:38917/ using document root <none> and password file <none>
I20260619 10:19:35.014222 14593 fs_manager.cc:359] Metadata directory not provided
I20260619 10:19:35.014266 14593 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 10:19:35.014420 14593 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 10:19:35.015499 14593 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781864224577058-825-0/minicluster-data/ts-0/data/instance:
uuid: "a392d075e14f4342808c63d2c776535d"
format_stamp: "Formatted at 2026-06-19 10:19:35 on dist-test-slave-6pt2"
server_key: "98abd49076517c583736948e0499b301"
server_key_iv: "dd592bbd5493ff87397663fbbc024616"
server_key_version: "encryptionkey@0"
I20260619 10:19:35.015839 14593 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781864224577058-825-0/minicluster-data/ts-0/wal/instance:
uuid: "a392d075e14f4342808c63d2c776535d"
format_stamp: "Formatted at 2026-06-19 10:19:35 on dist-test-slave-6pt2"
server_key: "98abd49076517c583736948e0499b301"
server_key_iv: "dd592bbd5493ff87397663fbbc024616"
server_key_version: "encryptionkey@0"
I20260619 10:19:35.017120 14593 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 10:19:35.017824 14608 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 10:19:35.018002 14593 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 10:19:35.018095 14593 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781864224577058-825-0/minicluster-data/ts-0/data,/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781864224577058-825-0/minicluster-data/ts-0/wal
uuid: "a392d075e14f4342808c63d2c776535d"
format_stamp: "Formatted at 2026-06-19 10:19:35 on dist-test-slave-6pt2"
server_key: "98abd49076517c583736948e0499b301"
server_key_iv: "dd592bbd5493ff87397663fbbc024616"
server_key_version: "encryptionkey@0"
I20260619 10:19:35.018177 14593 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781864224577058-825-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781864224577058-825-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781864224577058-825-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 10:19:35.029570 14593 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 10:19:35.031451 14593 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 10:19:35.031603 14593 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 10:19:35.031845 14593 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 10:19:35.032137 14593 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 10:19:35.032188 14593 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 10:19:35.032230 14593 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 10:19:35.032258 14593 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 10:19:35.037777 14593 rpc_server.cc:307] RPC server started. Bound to: 127.0.206.65:39665
I20260619 10:19:35.037873 14721 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.206.65:39665 every 8 connection(s)
I20260619 10:19:35.038224 14593 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781864224577058-825-0/minicluster-data/ts-0/data/info.pb
I20260619 10:19:35.042138 825 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTU9OP6/build/release/bin/kudu as pid 14593
I20260619 10:19:35.042244 825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781864224577058-825-0/minicluster-data/ts-0/wal/instance
I20260619 10:19:35.042464 825 external_mini_cluster.cc:1428] Setting key b281feba5c7b56721d1cbea42eb3992b
Jun 19 10:19:35 dist-test-slave-6pt2 krb5kdc[14486](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864375, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.65@KRBTEST.COM for kudu/127.0.206.126@KRBTEST.COM
I20260619 10:19:35.047652 14590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:19:35.038896 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.65:47391 (local address 127.0.206.126:36823)
0619 10:19:35.038943 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0619 10:19:35.038945 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 10:19:35.039177 (+ 232us) server_negotiation.cc:407] Connection header received
0619 10:19:35.039601 (+ 424us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:19:35.039602 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:19:35.039614 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:19:35.039662 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:19:35.040496 (+ 834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:19:35.040988 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:19:35.041915 (+ 927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:19:35.042267 (+ 352us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:19:35.042336 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:19:35.044650 (+ 2314us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 10:19:35.044651 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 10:19:35.044651 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 10:19:35.044657 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 10:19:35.046114 (+ 1457us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:19:35.046659 (+ 545us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:19:35.046661 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:19:35.046661 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:19:35.046711 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:19:35.046988 (+ 277us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:19:35.046988 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:19:35.046989 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:19:35.047103 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 10:19:35.047154 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0619 10:19:35.047515 (+ 361us) server_negotiation.cc:299] Negotiation successful
0619 10:19:35.047587 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260619 10:19:35.047930 14724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:19:35.038894 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:36823 (local address 127.0.206.65:47391)
0619 10:19:35.039064 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0619 10:19:35.039096 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0619 10:19:35.039526 (+ 430us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 10:19:35.039708 (+ 182us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 10:19:35.039711 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 10:19:35.040247 (+ 536us) client_negotiation.cc:189] Negotiated authn=SASL
0619 10:19:35.040418 (+ 171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:19:35.040422 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:19:35.041071 (+ 649us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:19:35.041072 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:19:35.041829 (+ 757us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:19:35.041831 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:19:35.042345 (+ 514us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:19:35.042347 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:19:35.042499 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:19:35.043082 (+ 583us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 10:19:35.043098 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0619 10:19:35.044554 (+ 1456us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 10:19:35.046215 (+ 1661us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 10:19:35.046223 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 10:19:35.046225 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 10:19:35.046555 (+ 330us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 10:19:35.046781 (+ 226us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 10:19:35.046782 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 10:19:35.046782 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 10:19:35.046920 (+ 138us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 10:19:35.047173 (+ 253us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 10:19:35.047180 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 10:19:35.047362 (+ 182us) client_negotiation.cc:769] Sending connection context
0619 10:19:35.047497 (+ 135us) client_negotiation.cc:240] Negotiation successful
0619 10:19:35.047669 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":87,"threads_started":1}
I20260619 10:19:35.048462 14722 heartbeater.cc:347] Connected to a master server at 127.0.206.126:36823
I20260619 10:19:35.048554 14722 heartbeater.cc:464] Registering TS with master...
I20260619 10:19:35.048755 14722 heartbeater.cc:511] Master 127.0.206.126:36823 requested a full tablet report, sending...
I20260619 10:19:35.049211 14531 ts_manager.cc:194] Registered new tserver with Master: a392d075e14f4342808c63d2c776535d (127.0.206.65:39665)
I20260619 10:19:35.049783 14531 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.206.65@KRBTEST.COM'} at 127.0.206.65:47391
WARNING: no policy specified for kudu/127.0.206.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.206.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.206.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781864224577058-825-0/krb5kdc/kudu_127.0.206.66.keytab.
Entry for principal kudu/127.0.206.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781864224577058-825-0/krb5kdc/kudu_127.0.206.66.keytab.
I20260619 10:19:35.065243 825 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTU9OP6/build/release/bin/kudu
/tmp/dist-test-taskTU9OP6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781864224577058-825-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781864224577058-825-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781864224577058-825-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781864224577058-825-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.206.66:0
--local_ip_for_outbound_sockets=127.0.206.66
--webserver_interface=127.0.206.66
--webserver_port=0
--tserver_master_addrs=127.0.206.126:36823
--keytab_file=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781864224577058-825-0/krb5kdc/kudu_127.0.206.66.keytab
--principal=kudu/127.0.206.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.206.84:42497
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781864224577058-825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781864224577058-825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781864224577058-825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 10:19:35.144388 14727 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 10:19:35.144600 14727 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 10:19:35.144624 14727 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 10:19:35.145776 14727 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 10:19:35.145815 14727 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 10:19:35.145870 14727 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.206.66
I20260619 10:19:35.147289 14727 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.206.84:42497
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781864224577058-825-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781864224577058-825-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781864224577058-825-0/krb5kdc/kudu_127.0.206.66.keytab
--principal=kudu/127.0.206.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.206.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781864224577058-825-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.206.66
--webserver_port=0
--tserver_master_addrs=127.0.206.126:36823
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14727
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.206.66
--log_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781864224577058-825-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 09:32:44 UTC on ccycloud.quasar-gbtldp.root.comops.site
build id 4455440
I20260619 10:19:35.147523 14727 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 10:19:35.147755 14727 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 10:19:35.152534 14735 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 10:19:35.152549 14733 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 10:19:35.152669 14727 server_base.cc:1034] running on GCE node
W20260619 10:19:35.152549 14732 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 10:19:35.152941 14727 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 10:19:35.153208 14727 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 10:19:35.154371 14727 hybrid_clock.cc:648] HybridClock initialized: now 1781864375154367 us; error 37 us; skew 500 ppm
Jun 19 10:19:35 dist-test-slave-6pt2 krb5kdc[14486](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864375, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 10:19:35.157061 14727 init.cc:377] Logged in from keytab as kudu/127.0.206.66@KRBTEST.COM (short username kudu)
I20260619 10:19:35.157473 14727 webserver.cc:460] Webserver started at http://127.0.206.66:45373/ using document root <none> and password file <none>
I20260619 10:19:35.157702 14727 fs_manager.cc:359] Metadata directory not provided
I20260619 10:19:35.157766 14727 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 10:19:35.157915 14727 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 10:19:35.159009 14727 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781864224577058-825-0/minicluster-data/ts-1/data/instance:
uuid: "83db006ab6a44dcebcf71eac5e82f47d"
format_stamp: "Formatted at 2026-06-19 10:19:35 on dist-test-slave-6pt2"
server_key: "34291ad4f48d2234c2443a0a8ef84bce"
server_key_iv: "d28f26ab00c975a1d8d7da5b6d8c0e31"
server_key_version: "encryptionkey@0"
I20260619 10:19:35.159384 14727 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781864224577058-825-0/minicluster-data/ts-1/wal/instance:
uuid: "83db006ab6a44dcebcf71eac5e82f47d"
format_stamp: "Formatted at 2026-06-19 10:19:35 on dist-test-slave-6pt2"
server_key: "34291ad4f48d2234c2443a0a8ef84bce"
server_key_iv: "d28f26ab00c975a1d8d7da5b6d8c0e31"
server_key_version: "encryptionkey@0"
I20260619 10:19:35.160672 14727 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 10:19:35.161372 14742 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 10:19:35.161538 14727 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 10:19:35.161633 14727 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781864224577058-825-0/minicluster-data/ts-1/data,/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781864224577058-825-0/minicluster-data/ts-1/wal
uuid: "83db006ab6a44dcebcf71eac5e82f47d"
format_stamp: "Formatted at 2026-06-19 10:19:35 on dist-test-slave-6pt2"
server_key: "34291ad4f48d2234c2443a0a8ef84bce"
server_key_iv: "d28f26ab00c975a1d8d7da5b6d8c0e31"
server_key_version: "encryptionkey@0"
I20260619 10:19:35.161710 14727 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781864224577058-825-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781864224577058-825-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781864224577058-825-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 10:19:35.179183 14727 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 10:19:35.181429 14727 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 10:19:35.181586 14727 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 10:19:35.181811 14727 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 10:19:35.182114 14727 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 10:19:35.182165 14727 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 10:19:35.182206 14727 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 10:19:35.182231 14727 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 10:19:35.187875 14727 rpc_server.cc:307] RPC server started. Bound to: 127.0.206.66:38041
I20260619 10:19:35.187916 14855 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.206.66:38041 every 8 connection(s)
I20260619 10:19:35.188222 14727 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781864224577058-825-0/minicluster-data/ts-1/data/info.pb
I20260619 10:19:35.189556 825 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTU9OP6/build/release/bin/kudu as pid 14727
I20260619 10:19:35.189663 825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781864224577058-825-0/minicluster-data/ts-1/wal/instance
I20260619 10:19:35.189860 825 external_mini_cluster.cc:1428] Setting key 1e0330fedea7081ee86e1020a4d261e4
Jun 19 10:19:35 dist-test-slave-6pt2 krb5kdc[14486](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864375, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.66@KRBTEST.COM for kudu/127.0.206.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.0.206.67@KRBTEST.COM; defaulting to no policy
I20260619 10:19:35.197679 14590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:19:35.188849 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.66:47659 (local address 127.0.206.126:36823)
0619 10:19:35.188889 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0619 10:19:35.188891 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 10:19:35.189109 (+ 218us) server_negotiation.cc:407] Connection header received
0619 10:19:35.189620 (+ 511us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:19:35.189621 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:19:35.189634 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:19:35.189700 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:19:35.190857 (+ 1157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:19:35.191357 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:19:35.192339 (+ 982us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:19:35.192539 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:19:35.192614 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:19:35.194968 (+ 2354us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 10:19:35.194970 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 10:19:35.194970 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 10:19:35.194978 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 10:19:35.196327 (+ 1349us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:19:35.196721 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:19:35.196722 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:19:35.196722 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:19:35.196781 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:19:35.197042 (+ 261us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:19:35.197043 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:19:35.197043 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:19:35.197163 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 10:19:35.197243 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0619 10:19:35.197507 (+ 264us) server_negotiation.cc:299] Negotiation successful
0619 10:19:35.197588 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260619 10:19:35.197914 14858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:19:35.188825 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:36823 (local address 127.0.206.66:47659)
0619 10:19:35.188971 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0619 10:19:35.189016 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0619 10:19:35.189512 (+ 496us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 10:19:35.189769 (+ 257us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 10:19:35.189773 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 10:19:35.190476 (+ 703us) client_negotiation.cc:189] Negotiated authn=SASL
0619 10:19:35.190756 (+ 280us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:19:35.190762 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:19:35.191449 (+ 687us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:19:35.191451 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:19:35.192249 (+ 798us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:19:35.192251 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:19:35.192620 (+ 369us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:19:35.192622 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:19:35.192755 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:19:35.193253 (+ 498us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 10:19:35.193262 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0619 10:19:35.194876 (+ 1614us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 10:19:35.196410 (+ 1534us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 10:19:35.196416 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 10:19:35.196419 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 10:19:35.196644 (+ 225us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 10:19:35.196847 (+ 203us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 10:19:35.196848 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 10:19:35.196848 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 10:19:35.196973 (+ 125us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 10:19:35.197233 (+ 260us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 10:19:35.197239 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 10:19:35.197375 (+ 136us) client_negotiation.cc:769] Sending connection context
0619 10:19:35.197475 (+ 100us) client_negotiation.cc:240] Negotiation successful
0619 10:19:35.197639 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":68,"threads_started":1}
I20260619 10:19:35.198336 14856 heartbeater.cc:347] Connected to a master server at 127.0.206.126:36823
I20260619 10:19:35.198446 14856 heartbeater.cc:464] Registering TS with master...
I20260619 10:19:35.198693 14856 heartbeater.cc:511] Master 127.0.206.126:36823 requested a full tablet report, sending...
I20260619 10:19:35.199076 14531 ts_manager.cc:194] Registered new tserver with Master: 83db006ab6a44dcebcf71eac5e82f47d (127.0.206.66:38041)
I20260619 10:19:35.199415 14531 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.206.66@KRBTEST.COM'} at 127.0.206.66:47659
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.206.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.206.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781864224577058-825-0/krb5kdc/kudu_127.0.206.67.keytab.
Entry for principal kudu/127.0.206.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781864224577058-825-0/krb5kdc/kudu_127.0.206.67.keytab.
I20260619 10:19:35.212621 825 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTU9OP6/build/release/bin/kudu
/tmp/dist-test-taskTU9OP6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781864224577058-825-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781864224577058-825-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781864224577058-825-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781864224577058-825-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.206.67:0
--local_ip_for_outbound_sockets=127.0.206.67
--webserver_interface=127.0.206.67
--webserver_port=0
--tserver_master_addrs=127.0.206.126:36823
--keytab_file=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781864224577058-825-0/krb5kdc/kudu_127.0.206.67.keytab
--principal=kudu/127.0.206.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.206.84:42497
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781864224577058-825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781864224577058-825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781864224577058-825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 10:19:35.291208 14861 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 10:19:35.291400 14861 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 10:19:35.291424 14861 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 10:19:35.292550 14861 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 10:19:35.292593 14861 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 10:19:35.292644 14861 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.206.67
I20260619 10:19:35.294061 14861 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.206.84:42497
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781864224577058-825-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781864224577058-825-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781864224577058-825-0/krb5kdc/kudu_127.0.206.67.keytab
--principal=kudu/127.0.206.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.206.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781864224577058-825-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.206.67
--webserver_port=0
--tserver_master_addrs=127.0.206.126:36823
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14861
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.206.67
--log_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781864224577058-825-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 09:32:44 UTC on ccycloud.quasar-gbtldp.root.comops.site
build id 4455440
I20260619 10:19:35.294287 14861 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 10:19:35.294497 14861 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 10:19:35.299312 14867 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 10:19:35.299355 14869 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 10:19:35.299335 14866 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 10:19:35.299615 14861 server_base.cc:1034] running on GCE node
I20260619 10:19:35.299819 14861 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 10:19:35.300082 14861 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 10:19:35.301247 14861 hybrid_clock.cc:648] HybridClock initialized: now 1781864375301243 us; error 36 us; skew 500 ppm
Jun 19 10:19:35 dist-test-slave-6pt2 krb5kdc[14486](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864375, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 10:19:35.303812 14861 init.cc:377] Logged in from keytab as kudu/127.0.206.67@KRBTEST.COM (short username kudu)
I20260619 10:19:35.304239 14861 webserver.cc:460] Webserver started at http://127.0.206.67:33311/ using document root <none> and password file <none>
I20260619 10:19:35.304420 14861 fs_manager.cc:359] Metadata directory not provided
I20260619 10:19:35.304458 14861 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 10:19:35.304543 14861 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 10:19:35.305531 14861 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781864224577058-825-0/minicluster-data/ts-2/data/instance:
uuid: "302e88a2fee04d3383b149616366acb0"
format_stamp: "Formatted at 2026-06-19 10:19:35 on dist-test-slave-6pt2"
server_key: "1ad1ae9fd44e12b3d5b32f86927da910"
server_key_iv: "61906ee126d7435d5887f96c7259397f"
server_key_version: "encryptionkey@0"
I20260619 10:19:35.305830 14861 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781864224577058-825-0/minicluster-data/ts-2/wal/instance:
uuid: "302e88a2fee04d3383b149616366acb0"
format_stamp: "Formatted at 2026-06-19 10:19:35 on dist-test-slave-6pt2"
server_key: "1ad1ae9fd44e12b3d5b32f86927da910"
server_key_iv: "61906ee126d7435d5887f96c7259397f"
server_key_version: "encryptionkey@0"
I20260619 10:19:35.307086 14861 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260619 10:19:35.307725 14876 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 10:19:35.307894 14861 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260619 10:19:35.307973 14861 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781864224577058-825-0/minicluster-data/ts-2/data,/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781864224577058-825-0/minicluster-data/ts-2/wal
uuid: "302e88a2fee04d3383b149616366acb0"
format_stamp: "Formatted at 2026-06-19 10:19:35 on dist-test-slave-6pt2"
server_key: "1ad1ae9fd44e12b3d5b32f86927da910"
server_key_iv: "61906ee126d7435d5887f96c7259397f"
server_key_version: "encryptionkey@0"
I20260619 10:19:35.308024 14861 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781864224577058-825-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781864224577058-825-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781864224577058-825-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 10:19:35.322091 14861 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 10:19:35.323804 14861 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 10:19:35.323907 14861 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 10:19:35.324066 14861 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 10:19:35.324344 14861 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 10:19:35.324375 14861 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 10:19:35.324396 14861 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 10:19:35.324445 14861 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 10:19:35.329993 14861 rpc_server.cc:307] RPC server started. Bound to: 127.0.206.67:43999
I20260619 10:19:35.330046 14989 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.206.67:43999 every 8 connection(s)
I20260619 10:19:35.330336 14861 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781864224577058-825-0/minicluster-data/ts-2/data/info.pb
Jun 19 10:19:35 dist-test-slave-6pt2 krb5kdc[14486](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864375, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.67@KRBTEST.COM for kudu/127.0.206.126@KRBTEST.COM
I20260619 10:19:35.336755 825 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTU9OP6/build/release/bin/kudu as pid 14861
I20260619 10:19:35.336838 825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781864224577058-825-0/minicluster-data/ts-2/wal/instance
I20260619 10:19:35.337041 825 external_mini_cluster.cc:1428] Setting key 30fb84b5fe643899ff9905acb857833a
I20260619 10:19:35.338994 14590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:19:35.331027 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.67:46137 (local address 127.0.206.126:36823)
0619 10:19:35.331066 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0619 10:19:35.331068 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 10:19:35.331297 (+ 229us) server_negotiation.cc:407] Connection header received
0619 10:19:35.331805 (+ 508us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:19:35.331806 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:19:35.331817 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:19:35.331867 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:19:35.332723 (+ 856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:19:35.333206 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:19:35.334087 (+ 881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:19:35.334260 (+ 173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:19:35.334307 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:19:35.336227 (+ 1920us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 10:19:35.336228 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 10:19:35.336229 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 10:19:35.336235 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 10:19:35.337871 (+ 1636us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:19:35.338202 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:19:35.338203 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:19:35.338203 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:19:35.338250 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:19:35.338444 (+ 194us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:19:35.338445 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:19:35.338445 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:19:35.338539 (+ 94us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 10:19:35.338588 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0619 10:19:35.338874 (+ 286us) server_negotiation.cc:299] Negotiation successful
0619 10:19:35.338933 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260619 10:19:35.339154 14992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:19:35.331032 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:36823 (local address 127.0.206.67:46137)
0619 10:19:35.331185 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0619 10:19:35.331217 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0619 10:19:35.331724 (+ 507us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 10:19:35.331918 (+ 194us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 10:19:35.331921 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 10:19:35.332468 (+ 547us) client_negotiation.cc:189] Negotiated authn=SASL
0619 10:19:35.332644 (+ 176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:19:35.332649 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:19:35.333282 (+ 633us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:19:35.333282 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:19:35.334020 (+ 738us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:19:35.334022 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:19:35.334326 (+ 304us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:19:35.334326 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:19:35.334417 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:19:35.334852 (+ 435us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 10:19:35.334859 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0619 10:19:35.336142 (+ 1283us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 10:19:35.337937 (+ 1795us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 10:19:35.337942 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 10:19:35.337943 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 10:19:35.338142 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 10:19:35.338306 (+ 164us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 10:19:35.338307 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 10:19:35.338307 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 10:19:35.338394 (+ 87us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 10:19:35.338591 (+ 197us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 10:19:35.338639 (+ 48us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 10:19:35.338759 (+ 120us) client_negotiation.cc:769] Sending connection context
0619 10:19:35.338842 (+ 83us) client_negotiation.cc:240] Negotiation successful
0619 10:19:35.338948 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":77,"threads_started":1}
I20260619 10:19:35.339615 14990 heartbeater.cc:347] Connected to a master server at 127.0.206.126:36823
I20260619 10:19:35.339708 14990 heartbeater.cc:464] Registering TS with master...
I20260619 10:19:35.339839 14990 heartbeater.cc:511] Master 127.0.206.126:36823 requested a full tablet report, sending...
I20260619 10:19:35.340148 14531 ts_manager.cc:194] Registered new tserver with Master: 302e88a2fee04d3383b149616366acb0 (127.0.206.67:43999)
I20260619 10:19:35.340463 14531 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.206.67@KRBTEST.COM'} at 127.0.206.67:46137
I20260619 10:19:35.348752 825 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 10:19:35.355571 14590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:19:35.350073 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49946 (local address 127.0.206.126:36823)
0619 10:19:35.350113 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0619 10:19:35.350114 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 10:19:35.350224 (+ 110us) server_negotiation.cc:407] Connection header received
0619 10:19:35.350263 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:19:35.350264 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:19:35.350271 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:19:35.350319 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:19:35.350909 (+ 590us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:19:35.351362 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:19:35.352035 (+ 673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:19:35.352207 (+ 172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:19:35.352255 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:19:35.353334 (+ 1079us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 10:19:35.353335 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 10:19:35.353335 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 10:19:35.353340 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0619 10:19:35.354683 (+ 1343us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:19:35.354975 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:19:35.354976 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:19:35.354976 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:19:35.355023 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:19:35.355180 (+ 157us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:19:35.355180 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:19:35.355180 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:19:35.355299 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 10:19:35.355367 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0619 10:19:35.355455 (+ 88us) server_negotiation.cc:299] Negotiation successful
0619 10:19:35.355511 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260619 10:19:35.357131 14531 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49946:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 10:19:35.358951 14531 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 10:19:35.365101 14999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:19:35.361196 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.65:39665 (local address 127.0.0.1:57016)
0619 10:19:35.361355 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0619 10:19:35.361394 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0619 10:19:35.361453 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 10:19:35.361938 (+ 485us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 10:19:35.361939 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 10:19:35.361941 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 10:19:35.362015 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:19:35.362017 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:19:35.363083 (+ 1066us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:19:35.363084 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:19:35.364050 (+ 966us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:19:35.364053 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:19:35.364768 (+ 715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:19:35.364769 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:19:35.364851 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:19:35.364888 (+ 37us) client_negotiation.cc:769] Sending connection context
0619 10:19:35.364965 (+ 77us) client_negotiation.cc:240] Negotiation successful
0619 10:19:35.365018 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":137,"mutex_wait_us":53,"spinlock_wait_cycles":768,"thread_start_us":59,"threads_started":1}
I20260619 10:19:35.365096 15002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:19:35.361381 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57016 (local address 127.0.206.65:39665)
0619 10:19:35.361795 (+ 414us) server_negotiation.cc:206] Beginning negotiation
0619 10:19:35.361798 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 10:19:35.361806 (+ 8us) server_negotiation.cc:407] Connection header received
0619 10:19:35.361836 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:19:35.361837 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:19:35.361849 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:19:35.361939 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 10:19:35.362103 (+ 164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:19:35.362985 (+ 882us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:19:35.364184 (+ 1199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:19:35.364673 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:19:35.364758 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:19:35.364818 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0619 10:19:35.364982 (+ 164us) server_negotiation.cc:299] Negotiation successful
0619 10:19:35.365018 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":374,"thread_start_us":80,"threads_started":1}
I20260619 10:19:35.365619 15001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:19:35.361196 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.67:43999 (local address 127.0.0.1:56272)
0619 10:19:35.361472 (+ 276us) negotiation.cc:107] Waiting for socket to connect
0619 10:19:35.361483 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 10:19:35.361540 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 10:19:35.361765 (+ 225us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 10:19:35.361768 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 10:19:35.361770 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 10:19:35.361830 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:19:35.361831 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:19:35.363129 (+ 1298us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:19:35.363130 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:19:35.364285 (+ 1155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:19:35.364288 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:19:35.365382 (+ 1094us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:19:35.365383 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:19:35.365455 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:19:35.365459 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 10:19:35.365516 (+ 57us) client_negotiation.cc:240] Negotiation successful
0619 10:19:35.365551 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":261,"mutex_wait_us":26,"thread_start_us":56,"threads_started":1}
I20260619 10:19:35.365669 15000 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:19:35.361285 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56272 (local address 127.0.206.67:43999)
0619 10:19:35.361578 (+ 293us) server_negotiation.cc:206] Beginning negotiation
0619 10:19:35.361584 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 10:19:35.361594 (+ 10us) server_negotiation.cc:407] Connection header received
0619 10:19:35.361631 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:19:35.361633 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:19:35.361658 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:19:35.361744 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 10:19:35.361886 (+ 142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:19:35.363017 (+ 1131us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:19:35.364372 (+ 1355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:19:35.365085 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:19:35.365141 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:19:35.365348 (+ 207us) server_negotiation.cc:1035] Waiting for connection context
0619 10:19:35.365548 (+ 200us) server_negotiation.cc:299] Negotiation successful
0619 10:19:35.365599 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":257,"thread_start_us":81,"threads_started":1}
I20260619 10:19:35.365870 14656 tablet_service.cc:1467] Processing CreateTablet for tablet aac525741ee1460eb52976a51f125f7c (DEFAULT_TABLE table=test-table [id=fd3249b2b8e54b5e863c263da1aacb47]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 10:19:35.366261 14656 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet aac525741ee1460eb52976a51f125f7c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 10:19:35.366364 15003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:19:35.361573 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47004 (local address 127.0.206.66:38041)
0619 10:19:35.361736 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0619 10:19:35.361742 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 10:19:35.362164 (+ 422us) server_negotiation.cc:407] Connection header received
0619 10:19:35.362210 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:19:35.362212 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:19:35.362227 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:19:35.362294 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 10:19:35.362481 (+ 187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:19:35.363558 (+ 1077us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:19:35.365120 (+ 1562us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:19:35.365921 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:19:35.365994 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:19:35.366044 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0619 10:19:35.366239 (+ 195us) server_negotiation.cc:299] Negotiation successful
0619 10:19:35.366288 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":129,"thread_start_us":96,"threads_started":1}
I20260619 10:19:35.366395 14998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:19:35.361379 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.66:38041 (local address 127.0.0.1:47004)
0619 10:19:35.362030 (+ 651us) negotiation.cc:107] Waiting for socket to connect
0619 10:19:35.362040 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 10:19:35.362086 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 10:19:35.362365 (+ 279us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 10:19:35.362366 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 10:19:35.362368 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 10:19:35.362401 (+ 33us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:19:35.362403 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:19:35.363680 (+ 1277us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:19:35.363681 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:19:35.365025 (+ 1344us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:19:35.365027 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:19:35.365995 (+ 968us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:19:35.365996 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:19:35.366112 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:19:35.366117 (+ 5us) client_negotiation.cc:769] Sending connection context
0619 10:19:35.366222 (+ 105us) client_negotiation.cc:240] Negotiation successful
0619 10:19:35.366261 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":638,"thread_start_us":56,"threads_started":1}
I20260619 10:19:35.366572 14924 tablet_service.cc:1467] Processing CreateTablet for tablet aac525741ee1460eb52976a51f125f7c (DEFAULT_TABLE table=test-table [id=fd3249b2b8e54b5e863c263da1aacb47]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 10:19:35.366952 14924 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet aac525741ee1460eb52976a51f125f7c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 10:19:35.367296 14790 tablet_service.cc:1467] Processing CreateTablet for tablet aac525741ee1460eb52976a51f125f7c (DEFAULT_TABLE table=test-table [id=fd3249b2b8e54b5e863c263da1aacb47]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 10:19:35.367585 14790 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet aac525741ee1460eb52976a51f125f7c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 10:19:35.369211 15004 tablet_bootstrap.cc:492] T aac525741ee1460eb52976a51f125f7c P 302e88a2fee04d3383b149616366acb0: Bootstrap starting.
I20260619 10:19:35.369827 15004 tablet_bootstrap.cc:654] T aac525741ee1460eb52976a51f125f7c P 302e88a2fee04d3383b149616366acb0: Neither blocks nor log segments found. Creating new log.
I20260619 10:19:35.369863 15005 tablet_bootstrap.cc:492] T aac525741ee1460eb52976a51f125f7c P a392d075e14f4342808c63d2c776535d: Bootstrap starting.
I20260619 10:19:35.370260 15004 log.cc:824] T aac525741ee1460eb52976a51f125f7c P 302e88a2fee04d3383b149616366acb0: Log is configured to *not* fsync() on all Append() calls
I20260619 10:19:35.370453 15005 tablet_bootstrap.cc:654] T aac525741ee1460eb52976a51f125f7c P a392d075e14f4342808c63d2c776535d: Neither blocks nor log segments found. Creating new log.
I20260619 10:19:35.371001 15004 tablet_bootstrap.cc:492] T aac525741ee1460eb52976a51f125f7c P 302e88a2fee04d3383b149616366acb0: No bootstrap required, opened a new log
I20260619 10:19:35.371057 15004 ts_tablet_manager.cc:1397] T aac525741ee1460eb52976a51f125f7c P 302e88a2fee04d3383b149616366acb0: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260619 10:19:35.371081 15005 log.cc:824] T aac525741ee1460eb52976a51f125f7c P a392d075e14f4342808c63d2c776535d: Log is configured to *not* fsync() on all Append() calls
I20260619 10:19:35.371748 15005 tablet_bootstrap.cc:492] T aac525741ee1460eb52976a51f125f7c P a392d075e14f4342808c63d2c776535d: No bootstrap required, opened a new log
I20260619 10:19:35.371824 15005 ts_tablet_manager.cc:1397] T aac525741ee1460eb52976a51f125f7c P a392d075e14f4342808c63d2c776535d: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260619 10:19:35.372774 15004 raft_consensus.cc:348] T aac525741ee1460eb52976a51f125f7c P 302e88a2fee04d3383b149616366acb0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a392d075e14f4342808c63d2c776535d" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 39665 } } peers { permanent_uuid: "302e88a2fee04d3383b149616366acb0" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 43999 } } peers { permanent_uuid: "83db006ab6a44dcebcf71eac5e82f47d" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 38041 } }
I20260619 10:19:35.372903 15004 raft_consensus.cc:374] T aac525741ee1460eb52976a51f125f7c P 302e88a2fee04d3383b149616366acb0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 10:19:35.372925 15004 raft_consensus.cc:729] T aac525741ee1460eb52976a51f125f7c P 302e88a2fee04d3383b149616366acb0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 302e88a2fee04d3383b149616366acb0, State: Initialized, Role: FOLLOWER
I20260619 10:19:35.373041 15004 consensus_queue.cc:260] T aac525741ee1460eb52976a51f125f7c P 302e88a2fee04d3383b149616366acb0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a392d075e14f4342808c63d2c776535d" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 39665 } } peers { permanent_uuid: "302e88a2fee04d3383b149616366acb0" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 43999 } } peers { permanent_uuid: "83db006ab6a44dcebcf71eac5e82f47d" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 38041 } }
I20260619 10:19:35.373342 15004 ts_tablet_manager.cc:1428] T aac525741ee1460eb52976a51f125f7c P 302e88a2fee04d3383b149616366acb0: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260619 10:19:35.373360 14990 heartbeater.cc:503] Master 127.0.206.126:36823 was elected leader, sending a full tablet report...
I20260619 10:19:35.373246 15005 raft_consensus.cc:348] T aac525741ee1460eb52976a51f125f7c P a392d075e14f4342808c63d2c776535d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a392d075e14f4342808c63d2c776535d" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 39665 } } peers { permanent_uuid: "302e88a2fee04d3383b149616366acb0" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 43999 } } peers { permanent_uuid: "83db006ab6a44dcebcf71eac5e82f47d" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 38041 } }
I20260619 10:19:35.373405 15005 raft_consensus.cc:374] T aac525741ee1460eb52976a51f125f7c P a392d075e14f4342808c63d2c776535d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 10:19:35.373450 15005 raft_consensus.cc:729] T aac525741ee1460eb52976a51f125f7c P a392d075e14f4342808c63d2c776535d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a392d075e14f4342808c63d2c776535d, State: Initialized, Role: FOLLOWER
I20260619 10:19:35.373530 15007 tablet_bootstrap.cc:492] T aac525741ee1460eb52976a51f125f7c P 83db006ab6a44dcebcf71eac5e82f47d: Bootstrap starting.
I20260619 10:19:35.373553 15005 consensus_queue.cc:260] T aac525741ee1460eb52976a51f125f7c P a392d075e14f4342808c63d2c776535d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a392d075e14f4342808c63d2c776535d" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 39665 } } peers { permanent_uuid: "302e88a2fee04d3383b149616366acb0" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 43999 } } peers { permanent_uuid: "83db006ab6a44dcebcf71eac5e82f47d" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 38041 } }
I20260619 10:19:35.373811 15005 ts_tablet_manager.cc:1428] T aac525741ee1460eb52976a51f125f7c P a392d075e14f4342808c63d2c776535d: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260619 10:19:35.373823 14722 heartbeater.cc:503] Master 127.0.206.126:36823 was elected leader, sending a full tablet report...
I20260619 10:19:35.374202 15007 tablet_bootstrap.cc:654] T aac525741ee1460eb52976a51f125f7c P 83db006ab6a44dcebcf71eac5e82f47d: Neither blocks nor log segments found. Creating new log.
I20260619 10:19:35.374799 15007 log.cc:824] T aac525741ee1460eb52976a51f125f7c P 83db006ab6a44dcebcf71eac5e82f47d: Log is configured to *not* fsync() on all Append() calls
I20260619 10:19:35.375411 15007 tablet_bootstrap.cc:492] T aac525741ee1460eb52976a51f125f7c P 83db006ab6a44dcebcf71eac5e82f47d: No bootstrap required, opened a new log
I20260619 10:19:35.375460 15007 ts_tablet_manager.cc:1397] T aac525741ee1460eb52976a51f125f7c P 83db006ab6a44dcebcf71eac5e82f47d: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260619 10:19:35.376884 15007 raft_consensus.cc:348] T aac525741ee1460eb52976a51f125f7c P 83db006ab6a44dcebcf71eac5e82f47d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a392d075e14f4342808c63d2c776535d" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 39665 } } peers { permanent_uuid: "302e88a2fee04d3383b149616366acb0" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 43999 } } peers { permanent_uuid: "83db006ab6a44dcebcf71eac5e82f47d" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 38041 } }
I20260619 10:19:35.377069 15007 raft_consensus.cc:374] T aac525741ee1460eb52976a51f125f7c P 83db006ab6a44dcebcf71eac5e82f47d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 10:19:35.377118 15007 raft_consensus.cc:729] T aac525741ee1460eb52976a51f125f7c P 83db006ab6a44dcebcf71eac5e82f47d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 83db006ab6a44dcebcf71eac5e82f47d, State: Initialized, Role: FOLLOWER
I20260619 10:19:35.377235 15007 consensus_queue.cc:260] T aac525741ee1460eb52976a51f125f7c P 83db006ab6a44dcebcf71eac5e82f47d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a392d075e14f4342808c63d2c776535d" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 39665 } } peers { permanent_uuid: "302e88a2fee04d3383b149616366acb0" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 43999 } } peers { permanent_uuid: "83db006ab6a44dcebcf71eac5e82f47d" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 38041 } }
I20260619 10:19:35.377496 15007 ts_tablet_manager.cc:1428] T aac525741ee1460eb52976a51f125f7c P 83db006ab6a44dcebcf71eac5e82f47d: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260619 10:19:35.377609 14856 heartbeater.cc:503] Master 127.0.206.126:36823 was elected leader, sending a full tablet report...
I20260619 10:19:35.420864 15010 raft_consensus.cc:482] T aac525741ee1460eb52976a51f125f7c P a392d075e14f4342808c63d2c776535d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 10:19:35.421027 15010 raft_consensus.cc:504] T aac525741ee1460eb52976a51f125f7c P a392d075e14f4342808c63d2c776535d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a392d075e14f4342808c63d2c776535d" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 39665 } } peers { permanent_uuid: "302e88a2fee04d3383b149616366acb0" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 43999 } } peers { permanent_uuid: "83db006ab6a44dcebcf71eac5e82f47d" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 38041 } }
I20260619 10:19:35.421456 15010 leader_election.cc:288] T aac525741ee1460eb52976a51f125f7c P a392d075e14f4342808c63d2c776535d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 302e88a2fee04d3383b149616366acb0 (127.0.206.67:43999), 83db006ab6a44dcebcf71eac5e82f47d (127.0.206.66:38041)
I20260619 10:19:35.424188 14724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:19:35.421624 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.67:43999 (local address 127.0.206.65:51729)
0619 10:19:35.421671 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0619 10:19:35.421684 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 10:19:35.421737 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 10:19:35.421875 (+ 138us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 10:19:35.421877 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 10:19:35.421878 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 10:19:35.421955 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:19:35.421957 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:19:35.422618 (+ 661us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:19:35.422619 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:19:35.423367 (+ 748us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:19:35.423369 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:19:35.423928 (+ 559us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:19:35.423929 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:19:35.424010 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:19:35.424013 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 10:19:35.424080 (+ 67us) client_negotiation.cc:240] Negotiation successful
0619 10:19:35.424118 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":23,"mutex_wait_us":4}
I20260619 10:19:35.424187 15000 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:19:35.421706 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.65:51729 (local address 127.0.206.67:43999)
0619 10:19:35.421751 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0619 10:19:35.421753 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 10:19:35.421763 (+ 10us) server_negotiation.cc:407] Connection header received
0619 10:19:35.421791 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:19:35.421792 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:19:35.421801 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:19:35.421849 (+ 48us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 10:19:35.422030 (+ 181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:19:35.422512 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:19:35.423464 (+ 952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:19:35.423846 (+ 382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:19:35.423932 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:19:35.423941 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0619 10:19:35.424085 (+ 144us) server_negotiation.cc:299] Negotiation successful
0619 10:19:35.424123 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260619 10:19:35.424446 15003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:19:35.421818 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.65:60497 (local address 127.0.206.66:38041)
0619 10:19:35.421875 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0619 10:19:35.421877 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 10:19:35.421983 (+ 106us) server_negotiation.cc:407] Connection header received
0619 10:19:35.422051 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:19:35.422053 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:19:35.422061 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:19:35.422140 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 10:19:35.422289 (+ 149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:19:35.422819 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:19:35.423703 (+ 884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:19:35.424066 (+ 363us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:19:35.424165 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:19:35.424172 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0619 10:19:35.424330 (+ 158us) server_negotiation.cc:299] Negotiation successful
0619 10:19:35.424377 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260619 10:19:35.424499 14944 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aac525741ee1460eb52976a51f125f7c" candidate_uuid: "a392d075e14f4342808c63d2c776535d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "302e88a2fee04d3383b149616366acb0" is_pre_election: true
I20260619 10:19:35.424638 14944 raft_consensus.cc:2454] T aac525741ee1460eb52976a51f125f7c P 302e88a2fee04d3383b149616366acb0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a392d075e14f4342808c63d2c776535d in term 0.
I20260619 10:19:35.424681 15013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:19:35.421722 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.66:38041 (local address 127.0.206.65:60497)
0619 10:19:35.421910 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0619 10:19:35.421919 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0619 10:19:35.421997 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 10:19:35.422146 (+ 149us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 10:19:35.422147 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 10:19:35.422148 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 10:19:35.422214 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:19:35.422216 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:19:35.422896 (+ 680us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:19:35.422897 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:19:35.423614 (+ 717us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:19:35.423616 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:19:35.424165 (+ 549us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:19:35.424165 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:19:35.424236 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:19:35.424241 (+ 5us) client_negotiation.cc:769] Sending connection context
0619 10:19:35.424537 (+ 296us) client_negotiation.cc:240] Negotiation successful
0619 10:19:35.424576 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":174,"thread_start_us":111,"threads_started":1}
I20260619 10:19:35.424983 14610 leader_election.cc:302] T aac525741ee1460eb52976a51f125f7c P a392d075e14f4342808c63d2c776535d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 302e88a2fee04d3383b149616366acb0, a392d075e14f4342808c63d2c776535d; no voters:
I20260619 10:19:35.425036 14810 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aac525741ee1460eb52976a51f125f7c" candidate_uuid: "a392d075e14f4342808c63d2c776535d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83db006ab6a44dcebcf71eac5e82f47d" is_pre_election: true
I20260619 10:19:35.425184 14810 raft_consensus.cc:2454] T aac525741ee1460eb52976a51f125f7c P 83db006ab6a44dcebcf71eac5e82f47d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a392d075e14f4342808c63d2c776535d in term 0.
I20260619 10:19:35.425158 15010 raft_consensus.cc:2781] T aac525741ee1460eb52976a51f125f7c P a392d075e14f4342808c63d2c776535d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 10:19:35.425225 15010 raft_consensus.cc:482] T aac525741ee1460eb52976a51f125f7c P a392d075e14f4342808c63d2c776535d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 10:19:35.425262 15010 raft_consensus.cc:3037] T aac525741ee1460eb52976a51f125f7c P a392d075e14f4342808c63d2c776535d [term 0 FOLLOWER]: Advancing to term 1
I20260619 10:19:35.426147 15010 raft_consensus.cc:504] T aac525741ee1460eb52976a51f125f7c P a392d075e14f4342808c63d2c776535d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a392d075e14f4342808c63d2c776535d" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 39665 } } peers { permanent_uuid: "302e88a2fee04d3383b149616366acb0" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 43999 } } peers { permanent_uuid: "83db006ab6a44dcebcf71eac5e82f47d" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 38041 } }
I20260619 10:19:35.426298 15010 leader_election.cc:288] T aac525741ee1460eb52976a51f125f7c P a392d075e14f4342808c63d2c776535d [CANDIDATE]: Term 1 election: Requested vote from peers 302e88a2fee04d3383b149616366acb0 (127.0.206.67:43999), 83db006ab6a44dcebcf71eac5e82f47d (127.0.206.66:38041)
I20260619 10:19:35.426472 14810 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aac525741ee1460eb52976a51f125f7c" candidate_uuid: "a392d075e14f4342808c63d2c776535d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83db006ab6a44dcebcf71eac5e82f47d"
I20260619 10:19:35.426476 14944 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aac525741ee1460eb52976a51f125f7c" candidate_uuid: "a392d075e14f4342808c63d2c776535d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "302e88a2fee04d3383b149616366acb0"
I20260619 10:19:35.426548 14810 raft_consensus.cc:3037] T aac525741ee1460eb52976a51f125f7c P 83db006ab6a44dcebcf71eac5e82f47d [term 0 FOLLOWER]: Advancing to term 1
I20260619 10:19:35.426548 14944 raft_consensus.cc:3037] T aac525741ee1460eb52976a51f125f7c P 302e88a2fee04d3383b149616366acb0 [term 0 FOLLOWER]: Advancing to term 1
I20260619 10:19:35.427448 14944 raft_consensus.cc:2454] T aac525741ee1460eb52976a51f125f7c P 302e88a2fee04d3383b149616366acb0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a392d075e14f4342808c63d2c776535d in term 1.
I20260619 10:19:35.427448 14810 raft_consensus.cc:2454] T aac525741ee1460eb52976a51f125f7c P 83db006ab6a44dcebcf71eac5e82f47d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a392d075e14f4342808c63d2c776535d in term 1.
I20260619 10:19:35.427639 14610 leader_election.cc:302] T aac525741ee1460eb52976a51f125f7c P a392d075e14f4342808c63d2c776535d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 302e88a2fee04d3383b149616366acb0, a392d075e14f4342808c63d2c776535d; no voters:
I20260619 10:19:35.427723 15010 raft_consensus.cc:2781] T aac525741ee1460eb52976a51f125f7c P a392d075e14f4342808c63d2c776535d [term 1 FOLLOWER]: Leader election won for term 1
I20260619 10:19:35.427868 15010 raft_consensus.cc:686] T aac525741ee1460eb52976a51f125f7c P a392d075e14f4342808c63d2c776535d [term 1 LEADER]: Becoming Leader. State: Replica: a392d075e14f4342808c63d2c776535d, State: Running, Role: LEADER
I20260619 10:19:35.427982 15010 consensus_queue.cc:237] T aac525741ee1460eb52976a51f125f7c P a392d075e14f4342808c63d2c776535d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a392d075e14f4342808c63d2c776535d" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 39665 } } peers { permanent_uuid: "302e88a2fee04d3383b149616366acb0" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 43999 } } peers { permanent_uuid: "83db006ab6a44dcebcf71eac5e82f47d" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 38041 } }
I20260619 10:19:35.428722 14531 catalog_manager.cc:5495] T aac525741ee1460eb52976a51f125f7c P a392d075e14f4342808c63d2c776535d reported cstate change: term changed from 0 to 1, leader changed from <none> to a392d075e14f4342808c63d2c776535d (127.0.206.65). New cstate: current_term: 1 leader_uuid: "a392d075e14f4342808c63d2c776535d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a392d075e14f4342808c63d2c776535d" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 39665 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "302e88a2fee04d3383b149616366acb0" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 43999 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "83db006ab6a44dcebcf71eac5e82f47d" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 38041 } health_report { overall_health: UNKNOWN } } }
W20260619 10:19:35.438719 14857 tablet.cc:2369] T aac525741ee1460eb52976a51f125f7c P 83db006ab6a44dcebcf71eac5e82f47d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 10:19:35.440316 15002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:19:35.437996 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57020 (local address 127.0.206.65:39665)
0619 10:19:35.438056 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0619 10:19:35.438058 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 10:19:35.438067 (+ 9us) server_negotiation.cc:407] Connection header received
0619 10:19:35.438103 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:19:35.438104 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:19:35.438118 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:19:35.438178 (+ 60us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 10:19:35.438283 (+ 105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:19:35.438807 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:19:35.439521 (+ 714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:19:35.439733 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:19:35.439803 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:19:35.439979 (+ 176us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 10:19:35.440083 (+ 104us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 10:19:35.440178 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0619 10:19:35.440205 (+ 27us) server_negotiation.cc:299] Negotiation successful
0619 10:19:35.440222 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260619 10:19:35.441890 14810 raft_consensus.cc:1261] T aac525741ee1460eb52976a51f125f7c P 83db006ab6a44dcebcf71eac5e82f47d [term 1 FOLLOWER]: Refusing update from remote peer a392d075e14f4342808c63d2c776535d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 10:19:35.441895 14944 raft_consensus.cc:1261] T aac525741ee1460eb52976a51f125f7c P 302e88a2fee04d3383b149616366acb0 [term 1 FOLLOWER]: Refusing update from remote peer a392d075e14f4342808c63d2c776535d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 10:19:35.442240 15014 consensus_queue.cc:1035] T aac525741ee1460eb52976a51f125f7c P a392d075e14f4342808c63d2c776535d [LEADER]: Connected to new peer: Peer: permanent_uuid: "83db006ab6a44dcebcf71eac5e82f47d" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 38041 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 10:19:35.442371 15010 consensus_queue.cc:1035] T aac525741ee1460eb52976a51f125f7c P a392d075e14f4342808c63d2c776535d [LEADER]: Connected to new peer: Peer: permanent_uuid: "302e88a2fee04d3383b149616366acb0" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 43999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 10:19:35.446688 15018 mvcc.cc:205] Tried to move back new op lower bound from 7298516481807540224 to 7298516481753505792. Current Snapshot: MvccSnapshot[applied={T|T < 7298516481807540224 or (T in {7298516481807540224})}]
I20260619 10:19:35.446991 14530 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49946:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 10:19:35.447086 14530 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49946:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 10:19:35.447760 14530 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 33de2c9a9d4148a281c87656d617ca65: Sending DeleteTablet for 3 replicas of tablet aac525741ee1460eb52976a51f125f7c
I20260619 10:19:35.448055 14924 tablet_service.cc:1514] Processing DeleteTablet for tablet aac525741ee1460eb52976a51f125f7c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 10:19:35 UTC) from {username='kudu', principal='kudu/127.0.206.126@KRBTEST.COM'} at 127.0.0.1:56272
I20260619 10:19:35.448105 14656 tablet_service.cc:1514] Processing DeleteTablet for tablet aac525741ee1460eb52976a51f125f7c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 10:19:35 UTC) from {username='kudu', principal='kudu/127.0.206.126@KRBTEST.COM'} at 127.0.0.1:57016
I20260619 10:19:35.448277 14790 tablet_service.cc:1514] Processing DeleteTablet for tablet aac525741ee1460eb52976a51f125f7c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 10:19:35 UTC) from {username='kudu', principal='kudu/127.0.206.126@KRBTEST.COM'} at 127.0.0.1:47004
I20260619 10:19:35.448477 15029 tablet_replica.cc:330] T aac525741ee1460eb52976a51f125f7c P 83db006ab6a44dcebcf71eac5e82f47d: stopping tablet replica
I20260619 10:19:35.448536 15028 tablet_replica.cc:330] T aac525741ee1460eb52976a51f125f7c P a392d075e14f4342808c63d2c776535d: stopping tablet replica
I20260619 10:19:35.448563 825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTU9OP6/build/release/bin/kudu with pid 14593
I20260619 10:19:35.448642 15028 raft_consensus.cc:2229] T aac525741ee1460eb52976a51f125f7c P a392d075e14f4342808c63d2c776535d [term 1 LEADER]: Raft consensus shutting down.
I20260619 10:19:35.448734 15028 raft_consensus.cc:2258] T aac525741ee1460eb52976a51f125f7c P a392d075e14f4342808c63d2c776535d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 10:19:35.449222 15027 tablet_replica.cc:330] T aac525741ee1460eb52976a51f125f7c P 302e88a2fee04d3383b149616366acb0: stopping tablet replica
I20260619 10:19:35.449307 15027 raft_consensus.cc:2229] T aac525741ee1460eb52976a51f125f7c P 302e88a2fee04d3383b149616366acb0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 10:19:35.449397 15027 raft_consensus.cc:2258] T aac525741ee1460eb52976a51f125f7c P 302e88a2fee04d3383b149616366acb0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 10:19:35.449884 15027 ts_tablet_manager.cc:1905] T aac525741ee1460eb52976a51f125f7c P 302e88a2fee04d3383b149616366acb0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 10:19:35.451316 15027 ts_tablet_manager.cc:1918] T aac525741ee1460eb52976a51f125f7c P 302e88a2fee04d3383b149616366acb0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 10:19:35.451380 15027 log.cc:1196] T aac525741ee1460eb52976a51f125f7c P 302e88a2fee04d3383b149616366acb0: Deleting WAL directory at /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781864224577058-825-0/minicluster-data/ts-2/wal/wals/aac525741ee1460eb52976a51f125f7c
I20260619 10:19:35.451655 15027 ts_tablet_manager.cc:1939] T aac525741ee1460eb52976a51f125f7c P 302e88a2fee04d3383b149616366acb0: Deleting consensus metadata
I20260619 10:19:35.452024 14516 catalog_manager.cc:4841] TS 302e88a2fee04d3383b149616366acb0 (127.0.206.67:43999): tablet aac525741ee1460eb52976a51f125f7c (table test-table [id=fd3249b2b8e54b5e863c263da1aacb47]) successfully deleted
W20260619 10:19:35.454175 14517 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.0.206.65:39665 (error 108)
I20260619 10:19:35.454437 15029 raft_consensus.cc:2229] T aac525741ee1460eb52976a51f125f7c P 83db006ab6a44dcebcf71eac5e82f47d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 10:19:35.454452 825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTU9OP6/build/release/bin/kudu with pid 14727
I20260619 10:19:35.454561 15029 raft_consensus.cc:2258] T aac525741ee1460eb52976a51f125f7c P 83db006ab6a44dcebcf71eac5e82f47d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 10:19:35.454649 14999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:19:35.454509 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.65:39665 (local address 127.0.0.1:57032)
0619 10:19:35.454558 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0619 10:19:35.454618 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.65:39665: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260619 10:19:35.458738 14517 catalog_manager.cc:4584] TS a392d075e14f4342808c63d2c776535d (127.0.206.65:39665): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet aac525741ee1460eb52976a51f125f7c: Network error: Client connection negotiation failed: client connection to 127.0.206.65:39665: connect: Connection refused (error 111)
I20260619 10:19:35.459918 14999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:19:35.459792 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.66:38041 (local address 127.0.0.1:47008)
0619 10:19:35.459856 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0619 10:19:35.459888 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.66:38041: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
I20260619 10:19:35.459906 825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTU9OP6/build/release/bin/kudu with pid 14861
I20260619 10:19:35.464285 825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTU9OP6/build/release/bin/kudu with pid 14499
I20260619 10:19:35.469108 825 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTU9OP6/build/release/bin/kudu
/tmp/dist-test-taskTU9OP6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781864224577058-825-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781864224577058-825-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781864224577058-825-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781864224577058-825-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.206.126:36823
--webserver_interface=127.0.206.126
--webserver_port=36799
--keytab_file=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781864224577058-825-0/krb5kdc/kudu_127.0.206.126.keytab
--principal=kudu/127.0.206.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.206.84:42497
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.206.126:36823
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781864224577058-825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781864224577058-825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781864224577058-825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 10:19:35.548921 15031 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 10:19:35.549209 15031 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 10:19:35.549268 15031 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 10:19:35.550479 15031 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 10:19:35.550539 15031 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 10:19:35.550562 15031 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 10:19:35.550591 15031 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 10:19:35.550633 15031 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 10:19:35.552091 15031 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.206.84:42497
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781864224577058-825-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781864224577058-825-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.0.206.126:36823
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781864224577058-825-0/krb5kdc/kudu_127.0.206.126.keytab
--principal=kudu/127.0.206.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.206.126:36823
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781864224577058-825-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.206.126
--webserver_port=36799
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15031
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781864224577058-825-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 09:32:44 UTC on ccycloud.quasar-gbtldp.root.comops.site
build id 4455440
I20260619 10:19:35.552359 15031 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 10:19:35.552584 15031 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 10:19:35.560865 15040 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 10:19:35.560861 15038 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 10:19:35.561002 15037 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 10:19:35.561095 15031 server_base.cc:1034] running on GCE node
I20260619 10:19:35.561343 15031 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 10:19:35.561652 15031 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 10:19:35.562811 15031 hybrid_clock.cc:648] HybridClock initialized: now 1781864375562806 us; error 37 us; skew 500 ppm
Jun 19 10:19:35 dist-test-slave-6pt2 krb5kdc[14486](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864375, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 10:19:35.565236 15031 init.cc:377] Logged in from keytab as kudu/127.0.206.126@KRBTEST.COM (short username kudu)
I20260619 10:19:35.565697 15031 webserver.cc:460] Webserver started at http://127.0.206.126:36799/ using document root <none> and password file <none>
I20260619 10:19:35.565932 15031 fs_manager.cc:359] Metadata directory not provided
I20260619 10:19:35.566011 15031 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 10:19:35.567480 15031 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 10:19:35.568110 15047 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 10:19:35.568274 15031 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 10:19:35.568377 15031 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781864224577058-825-0/minicluster-data/master-0/data,/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781864224577058-825-0/minicluster-data/master-0/wal
uuid: "33de2c9a9d4148a281c87656d617ca65"
format_stamp: "Formatted at 2026-06-19 10:19:34 on dist-test-slave-6pt2"
server_key: "df2feca8e810e6795a30dfc48cdad65a"
server_key_iv: "fdff8c532fcc333feada82355159374c"
server_key_version: "encryptionkey@0"
I20260619 10:19:35.568756 15031 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781864224577058-825-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781864224577058-825-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781864224577058-825-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 10:19:35.583011 15031 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 10:19:35.584748 15031 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 10:19:35.584897 15031 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 10:19:35.588722 15031 rpc_server.cc:307] RPC server started. Bound to: 127.0.206.126:36823
I20260619 10:19:35.588752 15099 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.206.126:36823 every 8 connection(s)
I20260619 10:19:35.589110 15031 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781864224577058-825-0/minicluster-data/master-0/data/info.pb
I20260619 10:19:35.590590 15100 sys_catalog.cc:263] Verifying existing consensus state
I20260619 10:19:35.591295 15100 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 33de2c9a9d4148a281c87656d617ca65: Bootstrap starting.
I20260619 10:19:35.593813 825 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTU9OP6/build/release/bin/kudu as pid 15031
I20260619 10:19:35.594074 825 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTU9OP6/build/release/bin/kudu
/tmp/dist-test-taskTU9OP6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781864224577058-825-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781864224577058-825-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781864224577058-825-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781864224577058-825-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.206.65:39665
--local_ip_for_outbound_sockets=127.0.206.65
--tserver_master_addrs=127.0.206.126:36823
--webserver_port=38917
--webserver_interface=127.0.206.65
--keytab_file=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781864224577058-825-0/krb5kdc/kudu_127.0.206.65.keytab
--principal=kudu/127.0.206.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.206.84:42497
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781864224577058-825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781864224577058-825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781864224577058-825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 10:19:35.599545 15100 log.cc:824] T 00000000000000000000000000000000 P 33de2c9a9d4148a281c87656d617ca65: Log is configured to *not* fsync() on all Append() calls
I20260619 10:19:35.601377 15100 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 33de2c9a9d4148a281c87656d617ca65: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260619 10:19:35.601677 15100 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 33de2c9a9d4148a281c87656d617ca65: Bootstrap complete.
I20260619 10:19:35.603935 15100 raft_consensus.cc:348] T 00000000000000000000000000000000 P 33de2c9a9d4148a281c87656d617ca65 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "33de2c9a9d4148a281c87656d617ca65" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 36823 } }
I20260619 10:19:35.604176 15100 raft_consensus.cc:729] T 00000000000000000000000000000000 P 33de2c9a9d4148a281c87656d617ca65 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 33de2c9a9d4148a281c87656d617ca65, State: Initialized, Role: FOLLOWER
I20260619 10:19:35.604385 15100 consensus_queue.cc:260] T 00000000000000000000000000000000 P 33de2c9a9d4148a281c87656d617ca65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "33de2c9a9d4148a281c87656d617ca65" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 36823 } }
I20260619 10:19:35.604480 15100 raft_consensus.cc:388] T 00000000000000000000000000000000 P 33de2c9a9d4148a281c87656d617ca65 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 10:19:35.604537 15100 raft_consensus.cc:482] T 00000000000000000000000000000000 P 33de2c9a9d4148a281c87656d617ca65 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 10:19:35.604586 15100 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 33de2c9a9d4148a281c87656d617ca65 [term 1 FOLLOWER]: Advancing to term 2
I20260619 10:19:35.605655 15100 raft_consensus.cc:504] T 00000000000000000000000000000000 P 33de2c9a9d4148a281c87656d617ca65 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "33de2c9a9d4148a281c87656d617ca65" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 36823 } }
I20260619 10:19:35.605770 15100 leader_election.cc:302] T 00000000000000000000000000000000 P 33de2c9a9d4148a281c87656d617ca65 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 33de2c9a9d4148a281c87656d617ca65; no voters:
I20260619 10:19:35.605945 15100 leader_election.cc:288] T 00000000000000000000000000000000 P 33de2c9a9d4148a281c87656d617ca65 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260619 10:19:35.606005 15104 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 33de2c9a9d4148a281c87656d617ca65 [term 2 FOLLOWER]: Leader election won for term 2
I20260619 10:19:35.606169 15104 raft_consensus.cc:686] T 00000000000000000000000000000000 P 33de2c9a9d4148a281c87656d617ca65 [term 2 LEADER]: Becoming Leader. State: Replica: 33de2c9a9d4148a281c87656d617ca65, State: Running, Role: LEADER
I20260619 10:19:35.606205 15100 sys_catalog.cc:564] T 00000000000000000000000000000000 P 33de2c9a9d4148a281c87656d617ca65 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 10:19:35.606285 15104 consensus_queue.cc:237] T 00000000000000000000000000000000 P 33de2c9a9d4148a281c87656d617ca65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "33de2c9a9d4148a281c87656d617ca65" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 36823 } }
I20260619 10:19:35.606536 15104 sys_catalog.cc:455] T 00000000000000000000000000000000 P 33de2c9a9d4148a281c87656d617ca65 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 33de2c9a9d4148a281c87656d617ca65. Latest consensus state: current_term: 2 leader_uuid: "33de2c9a9d4148a281c87656d617ca65" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "33de2c9a9d4148a281c87656d617ca65" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 36823 } } }
I20260619 10:19:35.606632 15104 sys_catalog.cc:458] T 00000000000000000000000000000000 P 33de2c9a9d4148a281c87656d617ca65 [sys.catalog]: This master's current role is: LEADER
I20260619 10:19:35.606540 15105 sys_catalog.cc:455] T 00000000000000000000000000000000 P 33de2c9a9d4148a281c87656d617ca65 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "33de2c9a9d4148a281c87656d617ca65" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "33de2c9a9d4148a281c87656d617ca65" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 36823 } } }
I20260619 10:19:35.606807 15110 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 10:19:35.606832 15105 sys_catalog.cc:458] T 00000000000000000000000000000000 P 33de2c9a9d4148a281c87656d617ca65 [sys.catalog]: This master's current role is: LEADER
I20260619 10:19:35.607584 15110 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 10:19:35.607679 15110 catalog_manager.cc:1259] Loaded cluster ID: ffbf713c1b564873b61b865becdfe3dd
I20260619 10:19:35.607712 15110 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 10:19:35.607956 15110 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260619 10:19:35.613226 15110 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260619 10:19:35.616127 15110 catalog_manager.cc:1504] Loading token signing keys...
I20260619 10:19:35.616307 15110 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 33de2c9a9d4148a281c87656d617ca65: Loaded TSK: 0
I20260619 10:19:35.616503 15110 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260619 10:19:35.677563 15102 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 10:19:35.677762 15102 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 10:19:35.677785 15102 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 10:19:35.678941 15102 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 10:19:35.678979 15102 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 10:19:35.679005 15102 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.206.65
I20260619 10:19:35.680400 15102 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.206.84:42497
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781864224577058-825-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781864224577058-825-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781864224577058-825-0/krb5kdc/kudu_127.0.206.65.keytab
--principal=kudu/127.0.206.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.206.65:39665
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781864224577058-825-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.206.65
--webserver_port=38917
--tserver_master_addrs=127.0.206.126:36823
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15102
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.206.65
--log_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781864224577058-825-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 09:32:44 UTC on ccycloud.quasar-gbtldp.root.comops.site
build id 4455440
I20260619 10:19:35.680626 15102 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 10:19:35.680859 15102 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 10:19:35.685626 15130 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 10:19:35.685636 15127 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 10:19:35.685636 15128 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 10:19:35.685885 15102 server_base.cc:1034] running on GCE node
I20260619 10:19:35.686085 15102 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 10:19:35.686357 15102 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 10:19:35.687541 15102 hybrid_clock.cc:648] HybridClock initialized: now 1781864375687520 us; error 36 us; skew 500 ppm
Jun 19 10:19:35 dist-test-slave-6pt2 krb5kdc[14486](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864375, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 10:19:35.690409 15102 init.cc:377] Logged in from keytab as kudu/127.0.206.65@KRBTEST.COM (short username kudu)
I20260619 10:19:35.690883 15102 webserver.cc:460] Webserver started at http://127.0.206.65:38917/ using document root <none> and password file <none>
I20260619 10:19:35.691125 15102 fs_manager.cc:359] Metadata directory not provided
I20260619 10:19:35.691207 15102 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 10:19:35.692545 15102 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 10:19:35.693252 15137 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 10:19:35.693416 15102 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 10:19:35.693506 15102 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781864224577058-825-0/minicluster-data/ts-0/data,/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781864224577058-825-0/minicluster-data/ts-0/wal
uuid: "a392d075e14f4342808c63d2c776535d"
format_stamp: "Formatted at 2026-06-19 10:19:35 on dist-test-slave-6pt2"
server_key: "98abd49076517c583736948e0499b301"
server_key_iv: "dd592bbd5493ff87397663fbbc024616"
server_key_version: "encryptionkey@0"
I20260619 10:19:35.693836 15102 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781864224577058-825-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781864224577058-825-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781864224577058-825-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 10:19:35.723727 15102 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 10:19:35.726011 15102 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 10:19:35.726174 15102 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 10:19:35.726419 15102 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 10:19:35.726933 15144 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260619 10:19:35.728350 15102 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260619 10:19:35.728408 15102 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260619 10:19:35.728435 15102 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260619 10:19:35.729079 15102 ts_tablet_manager.cc:610] Registered 1 tablets
I20260619 10:19:35.729130 15102 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260619 10:19:35.729162 15144 tablet_bootstrap.cc:492] T aac525741ee1460eb52976a51f125f7c P a392d075e14f4342808c63d2c776535d: Bootstrap starting.
I20260619 10:19:35.736589 15102 rpc_server.cc:307] RPC server started. Bound to: 127.0.206.65:39665
I20260619 10:19:35.736665 15251 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.206.65:39665 every 8 connection(s)
I20260619 10:19:35.737113 15102 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781864224577058-825-0/minicluster-data/ts-0/data/info.pb
I20260619 10:19:35.739042 825 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTU9OP6/build/release/bin/kudu as pid 15102
I20260619 10:19:35.739344 825 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTU9OP6/build/release/bin/kudu
/tmp/dist-test-taskTU9OP6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781864224577058-825-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781864224577058-825-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781864224577058-825-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781864224577058-825-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.206.66:38041
--local_ip_for_outbound_sockets=127.0.206.66
--tserver_master_addrs=127.0.206.126:36823
--webserver_port=45373
--webserver_interface=127.0.206.66
--keytab_file=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781864224577058-825-0/krb5kdc/kudu_127.0.206.66.keytab
--principal=kudu/127.0.206.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.206.84:42497
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781864224577058-825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781864224577058-825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781864224577058-825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 10:19:35.740496 15144 log.cc:824] T aac525741ee1460eb52976a51f125f7c P a392d075e14f4342808c63d2c776535d: Log is configured to *not* fsync() on all Append() calls
I20260619 10:19:35.742856 15144 tablet_bootstrap.cc:492] T aac525741ee1460eb52976a51f125f7c P a392d075e14f4342808c63d2c776535d: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260619 10:19:35.743162 15144 tablet_bootstrap.cc:492] T aac525741ee1460eb52976a51f125f7c P a392d075e14f4342808c63d2c776535d: Bootstrap complete.
I20260619 10:19:35.744303 15144 ts_tablet_manager.cc:1397] T aac525741ee1460eb52976a51f125f7c P a392d075e14f4342808c63d2c776535d: Time spent bootstrapping tablet: real 0.015s user 0.009s sys 0.004s
I20260619 10:19:35.745285 15144 raft_consensus.cc:348] T aac525741ee1460eb52976a51f125f7c P a392d075e14f4342808c63d2c776535d [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a392d075e14f4342808c63d2c776535d" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 39665 } } peers { permanent_uuid: "302e88a2fee04d3383b149616366acb0" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 43999 } } peers { permanent_uuid: "83db006ab6a44dcebcf71eac5e82f47d" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 38041 } }
I20260619 10:19:35.745604 15144 raft_consensus.cc:729] T aac525741ee1460eb52976a51f125f7c P a392d075e14f4342808c63d2c776535d [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: a392d075e14f4342808c63d2c776535d, State: Initialized, Role: FOLLOWER
I20260619 10:19:35.745740 15144 consensus_queue.cc:260] T aac525741ee1460eb52976a51f125f7c P a392d075e14f4342808c63d2c776535d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a392d075e14f4342808c63d2c776535d" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 39665 } } peers { permanent_uuid: "302e88a2fee04d3383b149616366acb0" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 43999 } } peers { permanent_uuid: "83db006ab6a44dcebcf71eac5e82f47d" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 38041 } }
I20260619 10:19:35.746003 15144 ts_tablet_manager.cc:1428] T aac525741ee1460eb52976a51f125f7c P a392d075e14f4342808c63d2c776535d: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
Jun 19 10:19:35 dist-test-slave-6pt2 krb5kdc[14486](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864375, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.65@KRBTEST.COM for kudu/127.0.206.126@KRBTEST.COM
I20260619 10:19:35.750176 15255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:19:35.737950 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.65:34273 (local address 127.0.206.126:36823)
0619 10:19:35.738097 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0619 10:19:35.738106 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 10:19:35.738166 (+ 60us) server_negotiation.cc:407] Connection header received
0619 10:19:35.739062 (+ 896us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:19:35.739069 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:19:35.739131 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:19:35.739277 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:19:35.740247 (+ 970us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:19:35.741445 (+ 1198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:19:35.743058 (+ 1613us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:19:35.743713 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:19:35.743823 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:19:35.746499 (+ 2676us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 10:19:35.746503 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 10:19:35.746503 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 10:19:35.746515 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0619 10:19:35.748267 (+ 1752us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:19:35.748826 (+ 559us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:19:35.748829 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:19:35.748829 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:19:35.748914 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:19:35.749211 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:19:35.749212 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:19:35.749212 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:19:35.749464 (+ 252us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 10:19:35.749537 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0619 10:19:35.749927 (+ 390us) server_negotiation.cc:299] Negotiation successful
0619 10:19:35.750040 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":106,"thread_start_us":77,"threads_started":1}
I20260619 10:19:35.750291 15254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:19:35.737802 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:36823 (local address 127.0.206.65:34273)
0619 10:19:35.738005 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0619 10:19:35.738057 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0619 10:19:35.738832 (+ 775us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 10:19:35.739323 (+ 491us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 10:19:35.739327 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 10:19:35.739821 (+ 494us) client_negotiation.cc:189] Negotiated authn=SASL
0619 10:19:35.740115 (+ 294us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:19:35.740122 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:19:35.741565 (+ 1443us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:19:35.741566 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:19:35.742976 (+ 1410us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:19:35.742979 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:19:35.743830 (+ 851us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:19:35.743831 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:19:35.743983 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:19:35.744623 (+ 640us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 10:19:35.744634 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0619 10:19:35.746388 (+ 1754us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 10:19:35.748353 (+ 1965us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 10:19:35.748360 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 10:19:35.748363 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 10:19:35.748720 (+ 357us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 10:19:35.748987 (+ 267us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 10:19:35.748987 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 10:19:35.748988 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 10:19:35.749139 (+ 151us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 10:19:35.749544 (+ 405us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 10:19:35.749551 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 10:19:35.749728 (+ 177us) client_negotiation.cc:769] Sending connection context
0619 10:19:35.749861 (+ 133us) client_negotiation.cc:240] Negotiation successful
0619 10:19:35.750044 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":104,"threads_started":1}
I20260619 10:19:35.751037 15252 heartbeater.cc:347] Connected to a master server at 127.0.206.126:36823
I20260619 10:19:35.751132 15252 heartbeater.cc:464] Registering TS with master...
I20260619 10:19:35.751300 15252 heartbeater.cc:511] Master 127.0.206.126:36823 requested a full tablet report, sending...
I20260619 10:19:35.752029 15064 ts_manager.cc:194] Registered new tserver with Master: a392d075e14f4342808c63d2c776535d (127.0.206.65:39665)
I20260619 10:19:35.753206 15064 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.206.65@KRBTEST.COM'} at 127.0.206.65:34273
I20260619 10:19:35.754106 15252 heartbeater.cc:503] Master 127.0.206.126:36823 was elected leader, sending a full tablet report...
Jun 19 10:19:35 dist-test-slave-6pt2 krb5kdc[14486](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864375, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.126@KRBTEST.COM for kudu/127.0.206.65@KRBTEST.COM
I20260619 10:19:35.761407 15259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:19:35.752619 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.65:39665 (local address 127.0.0.1:57040)
0619 10:19:35.752797 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0619 10:19:35.752830 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0619 10:19:35.752890 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 10:19:35.753640 (+ 750us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 10:19:35.753645 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 10:19:35.753880 (+ 235us) client_negotiation.cc:189] Negotiated authn=SASL
0619 10:19:35.753921 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:19:35.753923 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:19:35.755469 (+ 1546us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:19:35.755470 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:19:35.756065 (+ 595us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:19:35.756066 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:19:35.756649 (+ 583us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:19:35.756649 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:19:35.756707 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:19:35.757149 (+ 442us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 10:19:35.757153 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 10:19:35.758667 (+ 1514us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 10:19:35.760439 (+ 1772us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 10:19:35.760445 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 10:19:35.760447 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 10:19:35.760626 (+ 179us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 10:19:35.760831 (+ 205us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 10:19:35.760832 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 10:19:35.760832 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 10:19:35.760885 (+ 53us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 10:19:35.761149 (+ 264us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 10:19:35.761154 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 10:19:35.761199 (+ 45us) client_negotiation.cc:769] Sending connection context
0619 10:19:35.761260 (+ 61us) client_negotiation.cc:240] Negotiation successful
0619 10:19:35.761344 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":122,"threads_started":1}
I20260619 10:19:35.761475 15260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:19:35.752668 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57040 (local address 127.0.206.65:39665)
0619 10:19:35.753485 (+ 817us) server_negotiation.cc:206] Beginning negotiation
0619 10:19:35.753489 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 10:19:35.753499 (+ 10us) server_negotiation.cc:407] Connection header received
0619 10:19:35.753534 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:19:35.753536 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:19:35.753551 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:19:35.754276 (+ 725us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:19:35.754318 (+ 42us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:19:35.755393 (+ 1075us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:19:35.756141 (+ 748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:19:35.756576 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:19:35.756645 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:19:35.758753 (+ 2108us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 10:19:35.758760 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 10:19:35.758760 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 10:19:35.758767 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 10:19:35.760354 (+ 1587us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:19:35.760703 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:19:35.760706 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:19:35.760706 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:19:35.760765 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:19:35.760941 (+ 176us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:19:35.760942 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:19:35.760942 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:19:35.761081 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 10:19:35.761166 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0619 10:19:35.761306 (+ 140us) server_negotiation.cc:299] Negotiation successful
0619 10:19:35.761384 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":788,"thread_start_us":66,"threads_started":1}
I20260619 10:19:35.761806 15186 tablet_service.cc:1514] Processing DeleteTablet for tablet aac525741ee1460eb52976a51f125f7c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 10:19:35 UTC) from {username='kudu', principal='kudu/127.0.206.126@KRBTEST.COM'} at 127.0.0.1:57040
I20260619 10:19:35.762343 15261 tablet_replica.cc:330] stopping tablet replica
I20260619 10:19:35.762451 15261 raft_consensus.cc:2229] T aac525741ee1460eb52976a51f125f7c P a392d075e14f4342808c63d2c776535d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 10:19:35.762516 15261 raft_consensus.cc:2258] T aac525741ee1460eb52976a51f125f7c P a392d075e14f4342808c63d2c776535d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 10:19:35.762809 15261 ts_tablet_manager.cc:1905] T aac525741ee1460eb52976a51f125f7c P a392d075e14f4342808c63d2c776535d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 10:19:35.764729 15261 ts_tablet_manager.cc:1918] T aac525741ee1460eb52976a51f125f7c P a392d075e14f4342808c63d2c776535d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 10:19:35.764789 15261 log.cc:1196] T aac525741ee1460eb52976a51f125f7c P a392d075e14f4342808c63d2c776535d: Deleting WAL directory at /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781864224577058-825-0/minicluster-data/ts-0/wal/wals/aac525741ee1460eb52976a51f125f7c
I20260619 10:19:35.765012 15261 ts_tablet_manager.cc:1939] T aac525741ee1460eb52976a51f125f7c P a392d075e14f4342808c63d2c776535d: Deleting consensus metadata
I20260619 10:19:35.765475 15050 catalog_manager.cc:4841] TS a392d075e14f4342808c63d2c776535d (127.0.206.65:39665): tablet aac525741ee1460eb52976a51f125f7c (table test-table [id=fd3249b2b8e54b5e863c263da1aacb47]) successfully deleted
W20260619 10:19:35.825755 15256 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 10:19:35.825979 15256 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 10:19:35.826020 15256 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 10:19:35.827250 15256 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 10:19:35.827316 15256 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 10:19:35.827363 15256 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.206.66
I20260619 10:19:35.828838 15256 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.206.84:42497
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781864224577058-825-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781864224577058-825-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781864224577058-825-0/krb5kdc/kudu_127.0.206.66.keytab
--principal=kudu/127.0.206.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.206.66:38041
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781864224577058-825-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.206.66
--webserver_port=45373
--tserver_master_addrs=127.0.206.126:36823
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15256
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.206.66
--log_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781864224577058-825-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 09:32:44 UTC on ccycloud.quasar-gbtldp.root.comops.site
build id 4455440
I20260619 10:19:35.829129 15256 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 10:19:35.829353 15256 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 10:19:35.834162 15267 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 10:19:35.834146 15266 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 10:19:35.834435 15256 server_base.cc:1034] running on GCE node
W20260619 10:19:35.834134 15269 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 10:19:35.834733 15256 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 10:19:35.834971 15256 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 10:19:35.836128 15256 hybrid_clock.cc:648] HybridClock initialized: now 1781864375836119 us; error 27 us; skew 500 ppm
Jun 19 10:19:35 dist-test-slave-6pt2 krb5kdc[14486](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864375, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 10:19:35.838421 15256 init.cc:377] Logged in from keytab as kudu/127.0.206.66@KRBTEST.COM (short username kudu)
I20260619 10:19:35.838861 15256 webserver.cc:460] Webserver started at http://127.0.206.66:45373/ using document root <none> and password file <none>
I20260619 10:19:35.839041 15256 fs_manager.cc:359] Metadata directory not provided
I20260619 10:19:35.839089 15256 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 10:19:35.840279 15256 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 10:19:35.840870 15276 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 10:19:35.841053 15256 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 10:19:35.841118 15256 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781864224577058-825-0/minicluster-data/ts-1/data,/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781864224577058-825-0/minicluster-data/ts-1/wal
uuid: "83db006ab6a44dcebcf71eac5e82f47d"
format_stamp: "Formatted at 2026-06-19 10:19:35 on dist-test-slave-6pt2"
server_key: "34291ad4f48d2234c2443a0a8ef84bce"
server_key_iv: "d28f26ab00c975a1d8d7da5b6d8c0e31"
server_key_version: "encryptionkey@0"
I20260619 10:19:35.841395 15256 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781864224577058-825-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781864224577058-825-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781864224577058-825-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 10:19:35.852353 15256 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 10:19:35.853905 15256 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 10:19:35.854003 15256 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 10:19:35.854152 15256 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 10:19:35.854528 15283 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260619 10:19:35.855825 15256 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260619 10:19:35.855865 15256 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.001s user 0.000s sys 0.000s
I20260619 10:19:35.855887 15256 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260619 10:19:35.856447 15256 ts_tablet_manager.cc:610] Registered 1 tablets
I20260619 10:19:35.856477 15256 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260619 10:19:35.856568 15283 tablet_bootstrap.cc:492] T aac525741ee1460eb52976a51f125f7c P 83db006ab6a44dcebcf71eac5e82f47d: Bootstrap starting.
I20260619 10:19:35.863492 15256 rpc_server.cc:307] RPC server started. Bound to: 127.0.206.66:38041
I20260619 10:19:35.863584 15390 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.206.66:38041 every 8 connection(s)
I20260619 10:19:35.863888 15256 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781864224577058-825-0/minicluster-data/ts-1/data/info.pb
I20260619 10:19:35.864816 825 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTU9OP6/build/release/bin/kudu as pid 15256
I20260619 10:19:35.865114 825 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTU9OP6/build/release/bin/kudu
/tmp/dist-test-taskTU9OP6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781864224577058-825-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781864224577058-825-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781864224577058-825-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781864224577058-825-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.206.67:43999
--local_ip_for_outbound_sockets=127.0.206.67
--tserver_master_addrs=127.0.206.126:36823
--webserver_port=33311
--webserver_interface=127.0.206.67
--keytab_file=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781864224577058-825-0/krb5kdc/kudu_127.0.206.67.keytab
--principal=kudu/127.0.206.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.206.84:42497
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781864224577058-825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781864224577058-825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781864224577058-825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 10:19:35.865226 15283 log.cc:824] T aac525741ee1460eb52976a51f125f7c P 83db006ab6a44dcebcf71eac5e82f47d: Log is configured to *not* fsync() on all Append() calls
I20260619 10:19:35.867451 15283 tablet_bootstrap.cc:492] T aac525741ee1460eb52976a51f125f7c P 83db006ab6a44dcebcf71eac5e82f47d: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260619 10:19:35.867729 15283 tablet_bootstrap.cc:492] T aac525741ee1460eb52976a51f125f7c P 83db006ab6a44dcebcf71eac5e82f47d: Bootstrap complete.
I20260619 10:19:35.868784 15283 ts_tablet_manager.cc:1397] T aac525741ee1460eb52976a51f125f7c P 83db006ab6a44dcebcf71eac5e82f47d: Time spent bootstrapping tablet: real 0.012s user 0.001s sys 0.009s
I20260619 10:19:35.869979 15283 raft_consensus.cc:348] T aac525741ee1460eb52976a51f125f7c P 83db006ab6a44dcebcf71eac5e82f47d [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a392d075e14f4342808c63d2c776535d" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 39665 } } peers { permanent_uuid: "302e88a2fee04d3383b149616366acb0" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 43999 } } peers { permanent_uuid: "83db006ab6a44dcebcf71eac5e82f47d" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 38041 } }
I20260619 10:19:35.870237 15283 raft_consensus.cc:729] T aac525741ee1460eb52976a51f125f7c P 83db006ab6a44dcebcf71eac5e82f47d [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 83db006ab6a44dcebcf71eac5e82f47d, State: Initialized, Role: FOLLOWER
I20260619 10:19:35.870368 15283 consensus_queue.cc:260] T aac525741ee1460eb52976a51f125f7c P 83db006ab6a44dcebcf71eac5e82f47d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a392d075e14f4342808c63d2c776535d" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 39665 } } peers { permanent_uuid: "302e88a2fee04d3383b149616366acb0" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 43999 } } peers { permanent_uuid: "83db006ab6a44dcebcf71eac5e82f47d" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 38041 } }
I20260619 10:19:35.870676 15283 ts_tablet_manager.cc:1428] T aac525741ee1460eb52976a51f125f7c P 83db006ab6a44dcebcf71eac5e82f47d: Time spent starting tablet: real 0.002s user 0.003s sys 0.001s
Jun 19 10:19:35 dist-test-slave-6pt2 krb5kdc[14486](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864375, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.66@KRBTEST.COM for kudu/127.0.206.126@KRBTEST.COM
I20260619 10:19:35.875510 15393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:19:35.864510 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:36823 (local address 127.0.206.66:57241)
0619 10:19:35.865527 (+ 1017us) negotiation.cc:107] Waiting for socket to connect
0619 10:19:35.865569 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0619 10:19:35.866293 (+ 724us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 10:19:35.866648 (+ 355us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 10:19:35.866654 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 10:19:35.866996 (+ 342us) client_negotiation.cc:189] Negotiated authn=SASL
0619 10:19:35.867268 (+ 272us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:19:35.867274 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:19:35.868297 (+ 1023us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:19:35.868298 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:19:35.869604 (+ 1306us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:19:35.869607 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:19:35.870178 (+ 571us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:19:35.870180 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:19:35.870321 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:19:35.870958 (+ 637us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 10:19:35.870966 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 10:19:35.872412 (+ 1446us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 10:19:35.874238 (+ 1826us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 10:19:35.874244 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 10:19:35.874245 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 10:19:35.874451 (+ 206us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 10:19:35.874683 (+ 232us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 10:19:35.874684 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 10:19:35.874684 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 10:19:35.874790 (+ 106us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 10:19:35.875083 (+ 293us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 10:19:35.875088 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 10:19:35.875191 (+ 103us) client_negotiation.cc:769] Sending connection context
0619 10:19:35.875270 (+ 79us) client_negotiation.cc:240] Negotiation successful
0619 10:19:35.875382 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":975,"thread_start_us":106,"threads_started":1}
I20260619 10:19:35.875547 15255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:19:35.864604 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.66:57241 (local address 127.0.206.126:36823)
0619 10:19:35.865454 (+ 850us) server_negotiation.cc:206] Beginning negotiation
0619 10:19:35.865457 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 10:19:35.865649 (+ 192us) server_negotiation.cc:407] Connection header received
0619 10:19:35.866415 (+ 766us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:19:35.866417 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:19:35.866435 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:19:35.866502 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:19:35.867385 (+ 883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:19:35.868200 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:19:35.869705 (+ 1505us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:19:35.870081 (+ 376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:19:35.870164 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:19:35.872504 (+ 2340us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 10:19:35.872506 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 10:19:35.872506 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 10:19:35.872515 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0619 10:19:35.874158 (+ 1643us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:19:35.874528 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:19:35.874529 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:19:35.874530 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:19:35.874595 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:19:35.874859 (+ 264us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:19:35.874860 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:19:35.874861 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:19:35.875012 (+ 151us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 10:19:35.875091 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0619 10:19:35.875321 (+ 230us) server_negotiation.cc:299] Negotiation successful
0619 10:19:35.875416 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":798}
I20260619 10:19:35.875953 15391 heartbeater.cc:347] Connected to a master server at 127.0.206.126:36823
I20260619 10:19:35.876060 15391 heartbeater.cc:464] Registering TS with master...
I20260619 10:19:35.876266 15391 heartbeater.cc:511] Master 127.0.206.126:36823 requested a full tablet report, sending...
I20260619 10:19:35.876845 15064 ts_manager.cc:194] Registered new tserver with Master: 83db006ab6a44dcebcf71eac5e82f47d (127.0.206.66:38041)
I20260619 10:19:35.877360 15064 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.206.66@KRBTEST.COM'} at 127.0.206.66:57241
I20260619 10:19:35.878103 15391 heartbeater.cc:503] Master 127.0.206.126:36823 was elected leader, sending a full tablet report...
Jun 19 10:19:35 dist-test-slave-6pt2 krb5kdc[14486](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864375, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.126@KRBTEST.COM for kudu/127.0.206.66@KRBTEST.COM
I20260619 10:19:35.884107 15397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:19:35.877301 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47022 (local address 127.0.206.66:38041)
0619 10:19:35.877445 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0619 10:19:35.877449 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 10:19:35.877459 (+ 10us) server_negotiation.cc:407] Connection header received
0619 10:19:35.877492 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:19:35.877494 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:19:35.877508 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:19:35.877611 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:19:35.877861 (+ 250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:19:35.878765 (+ 904us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:19:35.879430 (+ 665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:19:35.879727 (+ 297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:19:35.879806 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:19:35.881522 (+ 1716us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 10:19:35.881528 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 10:19:35.881528 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 10:19:35.881534 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 10:19:35.883083 (+ 1549us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:19:35.883445 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:19:35.883447 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:19:35.883447 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:19:35.883495 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:19:35.883674 (+ 179us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:19:35.883675 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:19:35.883675 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:19:35.883789 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 10:19:35.883847 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0619 10:19:35.883988 (+ 141us) server_negotiation.cc:299] Negotiation successful
0619 10:19:35.884051 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":114,"thread_start_us":70,"threads_started":1}
I20260619 10:19:35.884146 15259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:19:35.877136 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.66:38041 (local address 127.0.0.1:47022)
0619 10:19:35.877182 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0619 10:19:35.877192 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 10:19:35.877225 (+ 33us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 10:19:35.877606 (+ 381us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 10:19:35.877606 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 10:19:35.877752 (+ 146us) client_negotiation.cc:189] Negotiated authn=SASL
0619 10:19:35.877783 (+ 31us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:19:35.877785 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:19:35.878847 (+ 1062us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:19:35.878848 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:19:35.879340 (+ 492us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:19:35.879342 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:19:35.879804 (+ 462us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:19:35.879805 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:19:35.879867 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:19:35.880271 (+ 404us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 10:19:35.880276 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 10:19:35.881439 (+ 1163us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 10:19:35.883159 (+ 1720us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 10:19:35.883160 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 10:19:35.883160 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 10:19:35.883367 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 10:19:35.883557 (+ 190us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 10:19:35.883558 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 10:19:35.883558 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 10:19:35.883618 (+ 60us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 10:19:35.883850 (+ 232us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 10:19:35.883850 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 10:19:35.883892 (+ 42us) client_negotiation.cc:769] Sending connection context
0619 10:19:35.883966 (+ 74us) client_negotiation.cc:240] Negotiation successful
0619 10:19:35.884054 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":31}
I20260619 10:19:35.884593 15325 tablet_service.cc:1514] Processing DeleteTablet for tablet aac525741ee1460eb52976a51f125f7c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 10:19:35 UTC) from {username='kudu', principal='kudu/127.0.206.126@KRBTEST.COM'} at 127.0.0.1:47022
I20260619 10:19:35.885001 15398 tablet_replica.cc:330] stopping tablet replica
I20260619 10:19:35.885110 15398 raft_consensus.cc:2229] T aac525741ee1460eb52976a51f125f7c P 83db006ab6a44dcebcf71eac5e82f47d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 10:19:35.885174 15398 raft_consensus.cc:2258] T aac525741ee1460eb52976a51f125f7c P 83db006ab6a44dcebcf71eac5e82f47d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 10:19:35.885432 15398 ts_tablet_manager.cc:1905] T aac525741ee1460eb52976a51f125f7c P 83db006ab6a44dcebcf71eac5e82f47d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 10:19:35.887166 15398 ts_tablet_manager.cc:1918] T aac525741ee1460eb52976a51f125f7c P 83db006ab6a44dcebcf71eac5e82f47d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 10:19:35.887223 15398 log.cc:1196] T aac525741ee1460eb52976a51f125f7c P 83db006ab6a44dcebcf71eac5e82f47d: Deleting WAL directory at /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781864224577058-825-0/minicluster-data/ts-1/wal/wals/aac525741ee1460eb52976a51f125f7c
I20260619 10:19:35.887481 15398 ts_tablet_manager.cc:1939] T aac525741ee1460eb52976a51f125f7c P 83db006ab6a44dcebcf71eac5e82f47d: Deleting consensus metadata
I20260619 10:19:35.887900 15049 catalog_manager.cc:4841] TS 83db006ab6a44dcebcf71eac5e82f47d (127.0.206.66:38041): tablet aac525741ee1460eb52976a51f125f7c (table test-table [id=fd3249b2b8e54b5e863c263da1aacb47]) successfully deleted
W20260619 10:19:35.949450 15395 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 10:19:35.949666 15395 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 10:19:35.949707 15395 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 10:19:35.950956 15395 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 10:19:35.951027 15395 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 10:19:35.951077 15395 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.206.67
I20260619 10:19:35.952481 15395 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.206.84:42497
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781864224577058-825-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781864224577058-825-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781864224577058-825-0/krb5kdc/kudu_127.0.206.67.keytab
--principal=kudu/127.0.206.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.206.67:43999
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781864224577058-825-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.206.67
--webserver_port=33311
--tserver_master_addrs=127.0.206.126:36823
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15395
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.206.67
--log_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781864224577058-825-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 09:32:44 UTC on ccycloud.quasar-gbtldp.root.comops.site
build id 4455440
I20260619 10:19:35.952737 15395 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 10:19:35.952955 15395 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 10:19:35.957813 15404 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 10:19:35.957842 15403 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 10:19:35.957844 15395 server_base.cc:1034] running on GCE node
W20260619 10:19:35.957815 15406 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 10:19:35.958173 15395 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 10:19:35.958449 15395 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 10:19:35.959604 15395 hybrid_clock.cc:648] HybridClock initialized: now 1781864375959598 us; error 37 us; skew 500 ppm
Jun 19 10:19:35 dist-test-slave-6pt2 krb5kdc[14486](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864375, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 10:19:35.961926 15395 init.cc:377] Logged in from keytab as kudu/127.0.206.67@KRBTEST.COM (short username kudu)
I20260619 10:19:35.962374 15395 webserver.cc:460] Webserver started at http://127.0.206.67:33311/ using document root <none> and password file <none>
I20260619 10:19:35.962589 15395 fs_manager.cc:359] Metadata directory not provided
I20260619 10:19:35.962719 15395 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 10:19:35.963959 15395 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260619 10:19:35.964521 15413 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 10:19:35.964684 15395 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260619 10:19:35.964779 15395 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781864224577058-825-0/minicluster-data/ts-2/data,/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781864224577058-825-0/minicluster-data/ts-2/wal
uuid: "302e88a2fee04d3383b149616366acb0"
format_stamp: "Formatted at 2026-06-19 10:19:35 on dist-test-slave-6pt2"
server_key: "1ad1ae9fd44e12b3d5b32f86927da910"
server_key_iv: "61906ee126d7435d5887f96c7259397f"
server_key_version: "encryptionkey@0"
I20260619 10:19:35.965087 15395 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781864224577058-825-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781864224577058-825-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781864224577058-825-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 10:19:35.974028 15395 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 10:19:35.975754 15395 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 10:19:35.975903 15395 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 10:19:35.976109 15395 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 10:19:35.976397 15395 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 10:19:35.976449 15395 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 10:19:35.976490 15395 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 10:19:35.976517 15395 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 10:19:35.982203 15395 rpc_server.cc:307] RPC server started. Bound to: 127.0.206.67:43999
I20260619 10:19:35.982245 15526 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.206.67:43999 every 8 connection(s)
I20260619 10:19:35.982553 15395 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781864224577058-825-0/minicluster-data/ts-2/data/info.pb
Jun 19 10:19:35 dist-test-slave-6pt2 krb5kdc[14486](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864375, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.67@KRBTEST.COM for kudu/127.0.206.126@KRBTEST.COM
I20260619 10:19:35.990427 825 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTU9OP6/build/release/bin/kudu as pid 15395
I20260619 10:19:35.990883 15255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:19:35.983267 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.67:59203 (local address 127.0.206.126:36823)
0619 10:19:35.983312 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0619 10:19:35.983314 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 10:19:35.983523 (+ 209us) server_negotiation.cc:407] Connection header received
0619 10:19:35.983944 (+ 421us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:19:35.983945 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:19:35.983953 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:19:35.984006 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:19:35.984867 (+ 861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:19:35.985350 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:19:35.986252 (+ 902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:19:35.986436 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:19:35.986492 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:19:35.988384 (+ 1892us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 10:19:35.988385 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 10:19:35.988386 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 10:19:35.988392 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 10:19:35.989530 (+ 1138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:19:35.989872 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:19:35.989872 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:19:35.989873 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:19:35.989934 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:19:35.990134 (+ 200us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:19:35.990135 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:19:35.990135 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:19:35.990227 (+ 92us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 10:19:35.990276 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0619 10:19:35.990690 (+ 414us) server_negotiation.cc:299] Negotiation successful
0619 10:19:35.990775 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260619 10:19:35.991037 15529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:19:35.983246 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:36823 (local address 127.0.206.67:59203)
0619 10:19:35.983395 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0619 10:19:35.983434 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0619 10:19:35.983869 (+ 435us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 10:19:35.984053 (+ 184us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 10:19:35.984056 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 10:19:35.984615 (+ 559us) client_negotiation.cc:189] Negotiated authn=SASL
0619 10:19:35.984795 (+ 180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:19:35.984800 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:19:35.985410 (+ 610us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:19:35.985411 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:19:35.986170 (+ 759us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:19:35.986171 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:19:35.986511 (+ 340us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:19:35.986512 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:19:35.986626 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:19:35.987015 (+ 389us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 10:19:35.987022 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0619 10:19:35.988301 (+ 1279us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 10:19:35.989597 (+ 1296us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 10:19:35.989603 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 10:19:35.989604 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 10:19:35.989808 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 10:19:35.989992 (+ 184us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 10:19:35.989992 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 10:19:35.989992 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 10:19:35.990082 (+ 90us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 10:19:35.990289 (+ 207us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 10:19:35.990295 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 10:19:35.990452 (+ 157us) client_negotiation.cc:769] Sending connection context
0619 10:19:35.990636 (+ 184us) client_negotiation.cc:240] Negotiation successful
0619 10:19:35.990799 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":83,"threads_started":1}
I20260619 10:19:35.991540 15527 heartbeater.cc:347] Connected to a master server at 127.0.206.126:36823
I20260619 10:19:35.991634 15527 heartbeater.cc:464] Registering TS with master...
I20260619 10:19:35.991811 15527 heartbeater.cc:511] Master 127.0.206.126:36823 requested a full tablet report, sending...
I20260619 10:19:35.992165 15064 ts_manager.cc:194] Registered new tserver with Master: 302e88a2fee04d3383b149616366acb0 (127.0.206.67:43999)
I20260619 10:19:35.992533 15064 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.206.67@KRBTEST.COM'} at 127.0.206.67:59203
I20260619 10:19:35.997020 15531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:19:35.990826 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49954 (local address 127.0.206.126:36823)
0619 10:19:35.991043 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0619 10:19:35.991044 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 10:19:35.991052 (+ 8us) server_negotiation.cc:407] Connection header received
0619 10:19:35.991175 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:19:35.991176 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:19:35.991187 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:19:35.991242 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:19:35.991945 (+ 703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:19:35.992530 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:19:35.993281 (+ 751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:19:35.993490 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:19:35.993573 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:19:35.994736 (+ 1163us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 10:19:35.994738 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 10:19:35.994738 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 10:19:35.994745 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 10:19:35.995899 (+ 1154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:19:35.996255 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:19:35.996256 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:19:35.996257 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:19:35.996315 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:19:35.996512 (+ 197us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:19:35.996513 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:19:35.996513 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:19:35.996618 (+ 105us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 10:19:35.996789 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0619 10:19:35.996829 (+ 40us) server_negotiation.cc:299] Negotiation successful
0619 10:19:35.996907 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":90,"thread_start_us":65,"threads_started":1}
I20260619 10:19:35.997653 825 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 10:19:36.000512 15531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:19:35.997946 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49956 (local address 127.0.206.126:36823)
0619 10:19:35.997979 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0619 10:19:35.997980 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 10:19:35.998059 (+ 79us) server_negotiation.cc:407] Connection header received
0619 10:19:35.998147 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:19:35.998147 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:19:35.998169 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:19:35.998227 (+ 58us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 10:19:35.998309 (+ 82us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:19:35.998868 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:19:35.999945 (+ 1077us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:19:36.000120 (+ 175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:19:36.000171 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:19:36.000291 (+ 120us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 10:19:36.000369 (+ 78us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 10:19:36.000425 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0619 10:19:36.000455 (+ 30us) server_negotiation.cc:299] Negotiation successful
0619 10:19:36.000472 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260619 10:19:36.000747 15064 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:49956:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 10:19:36.002236 15064 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 10:19:36.004269 15325 tablet_service.cc:1467] Processing CreateTablet for tablet 8704c4b5976f419fa4e68e28ba8f3eca (DEFAULT_TABLE table=test-table [id=8a5cff0b3fe54d189d233697bd096456]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 10:19:36.004343 15186 tablet_service.cc:1467] Processing CreateTablet for tablet 8704c4b5976f419fa4e68e28ba8f3eca (DEFAULT_TABLE table=test-table [id=8a5cff0b3fe54d189d233697bd096456]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 10:19:36.004458 15325 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8704c4b5976f419fa4e68e28ba8f3eca. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 10:19:36.004540 15186 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8704c4b5976f419fa4e68e28ba8f3eca. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 10:19:36.005883 15283 tablet_bootstrap.cc:492] T 8704c4b5976f419fa4e68e28ba8f3eca P 83db006ab6a44dcebcf71eac5e82f47d: Bootstrap starting.
I20260619 10:19:36.005909 15144 tablet_bootstrap.cc:492] T 8704c4b5976f419fa4e68e28ba8f3eca P a392d075e14f4342808c63d2c776535d: Bootstrap starting.
I20260619 10:19:36.006213 15283 tablet_bootstrap.cc:654] T 8704c4b5976f419fa4e68e28ba8f3eca P 83db006ab6a44dcebcf71eac5e82f47d: Neither blocks nor log segments found. Creating new log.
I20260619 10:19:36.006220 15144 tablet_bootstrap.cc:654] T 8704c4b5976f419fa4e68e28ba8f3eca P a392d075e14f4342808c63d2c776535d: Neither blocks nor log segments found. Creating new log.
I20260619 10:19:36.006814 15144 tablet_bootstrap.cc:492] T 8704c4b5976f419fa4e68e28ba8f3eca P a392d075e14f4342808c63d2c776535d: No bootstrap required, opened a new log
I20260619 10:19:36.006814 15283 tablet_bootstrap.cc:492] T 8704c4b5976f419fa4e68e28ba8f3eca P 83db006ab6a44dcebcf71eac5e82f47d: No bootstrap required, opened a new log
I20260619 10:19:36.006867 15144 ts_tablet_manager.cc:1397] T 8704c4b5976f419fa4e68e28ba8f3eca P a392d075e14f4342808c63d2c776535d: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20260619 10:19:36.006867 15283 ts_tablet_manager.cc:1397] T 8704c4b5976f419fa4e68e28ba8f3eca P 83db006ab6a44dcebcf71eac5e82f47d: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260619 10:19:36.007229 15283 raft_consensus.cc:348] T 8704c4b5976f419fa4e68e28ba8f3eca P 83db006ab6a44dcebcf71eac5e82f47d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "302e88a2fee04d3383b149616366acb0" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 43999 } } peers { permanent_uuid: "83db006ab6a44dcebcf71eac5e82f47d" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 38041 } } peers { permanent_uuid: "a392d075e14f4342808c63d2c776535d" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 39665 } }
I20260619 10:19:36.007233 15144 raft_consensus.cc:348] T 8704c4b5976f419fa4e68e28ba8f3eca P a392d075e14f4342808c63d2c776535d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "302e88a2fee04d3383b149616366acb0" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 43999 } } peers { permanent_uuid: "83db006ab6a44dcebcf71eac5e82f47d" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 38041 } } peers { permanent_uuid: "a392d075e14f4342808c63d2c776535d" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 39665 } }
I20260619 10:19:36.007305 15283 raft_consensus.cc:374] T 8704c4b5976f419fa4e68e28ba8f3eca P 83db006ab6a44dcebcf71eac5e82f47d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 10:19:36.007306 15144 raft_consensus.cc:374] T 8704c4b5976f419fa4e68e28ba8f3eca P a392d075e14f4342808c63d2c776535d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 10:19:36.007349 15283 raft_consensus.cc:729] T 8704c4b5976f419fa4e68e28ba8f3eca P 83db006ab6a44dcebcf71eac5e82f47d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 83db006ab6a44dcebcf71eac5e82f47d, State: Initialized, Role: FOLLOWER
I20260619 10:19:36.007349 15144 raft_consensus.cc:729] T 8704c4b5976f419fa4e68e28ba8f3eca P a392d075e14f4342808c63d2c776535d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a392d075e14f4342808c63d2c776535d, State: Initialized, Role: FOLLOWER
I20260619 10:19:36.007395 15283 consensus_queue.cc:260] T 8704c4b5976f419fa4e68e28ba8f3eca P 83db006ab6a44dcebcf71eac5e82f47d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "302e88a2fee04d3383b149616366acb0" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 43999 } } peers { permanent_uuid: "83db006ab6a44dcebcf71eac5e82f47d" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 38041 } } peers { permanent_uuid: "a392d075e14f4342808c63d2c776535d" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 39665 } }
I20260619 10:19:36.007395 15144 consensus_queue.cc:260] T 8704c4b5976f419fa4e68e28ba8f3eca P a392d075e14f4342808c63d2c776535d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "302e88a2fee04d3383b149616366acb0" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 43999 } } peers { permanent_uuid: "83db006ab6a44dcebcf71eac5e82f47d" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 38041 } } peers { permanent_uuid: "a392d075e14f4342808c63d2c776535d" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 39665 } }
I20260619 10:19:36.007483 15144 ts_tablet_manager.cc:1428] T 8704c4b5976f419fa4e68e28ba8f3eca P a392d075e14f4342808c63d2c776535d: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260619 10:19:36.007483 15283 ts_tablet_manager.cc:1428] T 8704c4b5976f419fa4e68e28ba8f3eca P 83db006ab6a44dcebcf71eac5e82f47d: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260619 10:19:36.009277 15259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:19:36.003946 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.67:43999 (local address 127.0.0.1:56282)
0619 10:19:36.003999 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0619 10:19:36.004008 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0619 10:19:36.004041 (+ 33us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 10:19:36.006128 (+ 2087us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 10:19:36.006129 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 10:19:36.006130 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 10:19:36.006163 (+ 33us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:19:36.006165 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:19:36.007461 (+ 1296us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:19:36.007462 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:19:36.008412 (+ 950us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:19:36.008415 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:19:36.009027 (+ 612us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:19:36.009028 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:19:36.009085 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:19:36.009103 (+ 18us) client_negotiation.cc:769] Sending connection context
0619 10:19:36.009163 (+ 60us) client_negotiation.cc:240] Negotiation successful
0619 10:19:36.009218 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36}
I20260619 10:19:36.009285 15533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:19:36.004010 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56282 (local address 127.0.206.67:43999)
0619 10:19:36.005947 (+ 1937us) server_negotiation.cc:206] Beginning negotiation
0619 10:19:36.005951 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 10:19:36.005961 (+ 10us) server_negotiation.cc:407] Connection header received
0619 10:19:36.005996 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:19:36.005998 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:19:36.006013 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:19:36.006125 (+ 112us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 10:19:36.006217 (+ 92us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:19:36.007320 (+ 1103us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:19:36.008492 (+ 1172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:19:36.008944 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:19:36.009029 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:19:36.009082 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0619 10:19:36.009176 (+ 94us) server_negotiation.cc:299] Negotiation successful
0619 10:19:36.009214 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1905,"thread_start_us":107,"threads_started":1}
I20260619 10:19:36.009891 15461 tablet_service.cc:1467] Processing CreateTablet for tablet 8704c4b5976f419fa4e68e28ba8f3eca (DEFAULT_TABLE table=test-table [id=8a5cff0b3fe54d189d233697bd096456]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 10:19:36.010185 15461 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8704c4b5976f419fa4e68e28ba8f3eca. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 10:19:36.012915 15536 tablet_bootstrap.cc:492] T 8704c4b5976f419fa4e68e28ba8f3eca P 302e88a2fee04d3383b149616366acb0: Bootstrap starting.
I20260619 10:19:36.013504 15536 tablet_bootstrap.cc:654] T 8704c4b5976f419fa4e68e28ba8f3eca P 302e88a2fee04d3383b149616366acb0: Neither blocks nor log segments found. Creating new log.
I20260619 10:19:36.013974 15536 log.cc:824] T 8704c4b5976f419fa4e68e28ba8f3eca P 302e88a2fee04d3383b149616366acb0: Log is configured to *not* fsync() on all Append() calls
I20260619 10:19:36.014670 15536 tablet_bootstrap.cc:492] T 8704c4b5976f419fa4e68e28ba8f3eca P 302e88a2fee04d3383b149616366acb0: No bootstrap required, opened a new log
I20260619 10:19:36.014732 15536 ts_tablet_manager.cc:1397] T 8704c4b5976f419fa4e68e28ba8f3eca P 302e88a2fee04d3383b149616366acb0: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260619 10:19:36.016214 15536 raft_consensus.cc:348] T 8704c4b5976f419fa4e68e28ba8f3eca P 302e88a2fee04d3383b149616366acb0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "302e88a2fee04d3383b149616366acb0" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 43999 } } peers { permanent_uuid: "83db006ab6a44dcebcf71eac5e82f47d" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 38041 } } peers { permanent_uuid: "a392d075e14f4342808c63d2c776535d" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 39665 } }
I20260619 10:19:36.016355 15536 raft_consensus.cc:374] T 8704c4b5976f419fa4e68e28ba8f3eca P 302e88a2fee04d3383b149616366acb0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 10:19:36.016377 15536 raft_consensus.cc:729] T 8704c4b5976f419fa4e68e28ba8f3eca P 302e88a2fee04d3383b149616366acb0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 302e88a2fee04d3383b149616366acb0, State: Initialized, Role: FOLLOWER
I20260619 10:19:36.016525 15536 consensus_queue.cc:260] T 8704c4b5976f419fa4e68e28ba8f3eca P 302e88a2fee04d3383b149616366acb0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "302e88a2fee04d3383b149616366acb0" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 43999 } } peers { permanent_uuid: "83db006ab6a44dcebcf71eac5e82f47d" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 38041 } } peers { permanent_uuid: "a392d075e14f4342808c63d2c776535d" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 39665 } }
I20260619 10:19:36.016788 15527 heartbeater.cc:503] Master 127.0.206.126:36823 was elected leader, sending a full tablet report...
I20260619 10:19:36.016810 15536 ts_tablet_manager.cc:1428] T 8704c4b5976f419fa4e68e28ba8f3eca P 302e88a2fee04d3383b149616366acb0: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260619 10:19:36.074009 15396 raft_consensus.cc:482] T 8704c4b5976f419fa4e68e28ba8f3eca P 83db006ab6a44dcebcf71eac5e82f47d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 10:19:36.074136 15396 raft_consensus.cc:504] T 8704c4b5976f419fa4e68e28ba8f3eca P 83db006ab6a44dcebcf71eac5e82f47d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "302e88a2fee04d3383b149616366acb0" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 43999 } } peers { permanent_uuid: "83db006ab6a44dcebcf71eac5e82f47d" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 38041 } } peers { permanent_uuid: "a392d075e14f4342808c63d2c776535d" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 39665 } }
I20260619 10:19:36.074519 15396 leader_election.cc:288] T 8704c4b5976f419fa4e68e28ba8f3eca P 83db006ab6a44dcebcf71eac5e82f47d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 302e88a2fee04d3383b149616366acb0 (127.0.206.67:43999), a392d075e14f4342808c63d2c776535d (127.0.206.65:39665)
I20260619 10:19:36.078444 15260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:19:36.074772 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.66:44225 (local address 127.0.206.65:39665)
0619 10:19:36.074818 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0619 10:19:36.074820 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 10:19:36.074984 (+ 164us) server_negotiation.cc:407] Connection header received
0619 10:19:36.075020 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:19:36.075022 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:19:36.075047 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:19:36.075107 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 10:19:36.075259 (+ 152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:19:36.075997 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:19:36.077475 (+ 1478us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:19:36.078076 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:19:36.078139 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:19:36.078218 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0619 10:19:36.078332 (+ 114us) server_negotiation.cc:299] Negotiation successful
0619 10:19:36.078372 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260619 10:19:36.078444 15533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:19:36.074811 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.66:52355 (local address 127.0.206.67:43999)
0619 10:19:36.074906 (+ 95us) server_negotiation.cc:206] Beginning negotiation
0619 10:19:36.074908 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 10:19:36.074914 (+ 6us) server_negotiation.cc:407] Connection header received
0619 10:19:36.074936 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:19:36.074937 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:19:36.074967 (+ 30us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:19:36.075024 (+ 57us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 10:19:36.075247 (+ 223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:19:36.075998 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:19:36.077474 (+ 1476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:19:36.078049 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:19:36.078134 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:19:36.078144 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0619 10:19:36.078332 (+ 188us) server_negotiation.cc:299] Negotiation successful
0619 10:19:36.078372 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260619 10:19:36.078490 15539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:19:36.074703 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.65:39665 (local address 127.0.206.66:44225)
0619 10:19:36.074887 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0619 10:19:36.074897 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 10:19:36.074962 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 10:19:36.075126 (+ 164us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 10:19:36.075127 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 10:19:36.075129 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 10:19:36.075183 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:19:36.075184 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:19:36.076079 (+ 895us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:19:36.076080 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:19:36.077383 (+ 1303us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:19:36.077386 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:19:36.078161 (+ 775us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:19:36.078162 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:19:36.078227 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:19:36.078251 (+ 24us) client_negotiation.cc:769] Sending connection context
0619 10:19:36.078312 (+ 61us) client_negotiation.cc:240] Negotiation successful
0619 10:19:36.078366 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":168,"mutex_wait_us":47,"thread_start_us":95,"threads_started":1}
I20260619 10:19:36.078490 15393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:19:36.074700 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.67:43999 (local address 127.0.206.66:52355)
0619 10:19:36.074744 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0619 10:19:36.074755 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 10:19:36.074808 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 10:19:36.075054 (+ 246us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 10:19:36.075055 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 10:19:36.075057 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 10:19:36.075150 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:19:36.075152 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:19:36.076079 (+ 927us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:19:36.076079 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:19:36.077383 (+ 1304us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:19:36.077386 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:19:36.078148 (+ 762us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:19:36.078149 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:19:36.078232 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:19:36.078251 (+ 19us) client_negotiation.cc:769] Sending connection context
0619 10:19:36.078312 (+ 61us) client_negotiation.cc:240] Negotiation successful
0619 10:19:36.078366 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":26,"mutex_wait_us":97}
I20260619 10:19:36.078804 15206 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8704c4b5976f419fa4e68e28ba8f3eca" candidate_uuid: "83db006ab6a44dcebcf71eac5e82f47d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a392d075e14f4342808c63d2c776535d" is_pre_election: true
I20260619 10:19:36.078966 15206 raft_consensus.cc:2454] T 8704c4b5976f419fa4e68e28ba8f3eca P a392d075e14f4342808c63d2c776535d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83db006ab6a44dcebcf71eac5e82f47d in term 0.
I20260619 10:19:36.079041 15481 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8704c4b5976f419fa4e68e28ba8f3eca" candidate_uuid: "83db006ab6a44dcebcf71eac5e82f47d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "302e88a2fee04d3383b149616366acb0" is_pre_election: true
I20260619 10:19:36.079169 15481 raft_consensus.cc:2454] T 8704c4b5976f419fa4e68e28ba8f3eca P 302e88a2fee04d3383b149616366acb0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83db006ab6a44dcebcf71eac5e82f47d in term 0.
I20260619 10:19:36.079360 15278 leader_election.cc:302] T 8704c4b5976f419fa4e68e28ba8f3eca P 83db006ab6a44dcebcf71eac5e82f47d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 302e88a2fee04d3383b149616366acb0, 83db006ab6a44dcebcf71eac5e82f47d; no voters:
I20260619 10:19:36.079530 15396 raft_consensus.cc:2781] T 8704c4b5976f419fa4e68e28ba8f3eca P 83db006ab6a44dcebcf71eac5e82f47d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 10:19:36.079584 15396 raft_consensus.cc:482] T 8704c4b5976f419fa4e68e28ba8f3eca P 83db006ab6a44dcebcf71eac5e82f47d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 10:19:36.079602 15396 raft_consensus.cc:3037] T 8704c4b5976f419fa4e68e28ba8f3eca P 83db006ab6a44dcebcf71eac5e82f47d [term 0 FOLLOWER]: Advancing to term 1
I20260619 10:19:36.080317 15396 raft_consensus.cc:504] T 8704c4b5976f419fa4e68e28ba8f3eca P 83db006ab6a44dcebcf71eac5e82f47d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "302e88a2fee04d3383b149616366acb0" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 43999 } } peers { permanent_uuid: "83db006ab6a44dcebcf71eac5e82f47d" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 38041 } } peers { permanent_uuid: "a392d075e14f4342808c63d2c776535d" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 39665 } }
I20260619 10:19:36.080457 15396 leader_election.cc:288] T 8704c4b5976f419fa4e68e28ba8f3eca P 83db006ab6a44dcebcf71eac5e82f47d [CANDIDATE]: Term 1 election: Requested vote from peers 302e88a2fee04d3383b149616366acb0 (127.0.206.67:43999), a392d075e14f4342808c63d2c776535d (127.0.206.65:39665)
I20260619 10:19:36.080618 15481 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8704c4b5976f419fa4e68e28ba8f3eca" candidate_uuid: "83db006ab6a44dcebcf71eac5e82f47d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "302e88a2fee04d3383b149616366acb0"
I20260619 10:19:36.080613 15206 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8704c4b5976f419fa4e68e28ba8f3eca" candidate_uuid: "83db006ab6a44dcebcf71eac5e82f47d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a392d075e14f4342808c63d2c776535d"
I20260619 10:19:36.080689 15206 raft_consensus.cc:3037] T 8704c4b5976f419fa4e68e28ba8f3eca P a392d075e14f4342808c63d2c776535d [term 0 FOLLOWER]: Advancing to term 1
I20260619 10:19:36.080690 15481 raft_consensus.cc:3037] T 8704c4b5976f419fa4e68e28ba8f3eca P 302e88a2fee04d3383b149616366acb0 [term 0 FOLLOWER]: Advancing to term 1
I20260619 10:19:36.081509 15206 raft_consensus.cc:2454] T 8704c4b5976f419fa4e68e28ba8f3eca P a392d075e14f4342808c63d2c776535d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83db006ab6a44dcebcf71eac5e82f47d in term 1.
I20260619 10:19:36.081509 15481 raft_consensus.cc:2454] T 8704c4b5976f419fa4e68e28ba8f3eca P 302e88a2fee04d3383b149616366acb0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83db006ab6a44dcebcf71eac5e82f47d in term 1.
I20260619 10:19:36.081687 15279 leader_election.cc:302] T 8704c4b5976f419fa4e68e28ba8f3eca P 83db006ab6a44dcebcf71eac5e82f47d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83db006ab6a44dcebcf71eac5e82f47d, a392d075e14f4342808c63d2c776535d; no voters:
I20260619 10:19:36.081811 15396 raft_consensus.cc:2781] T 8704c4b5976f419fa4e68e28ba8f3eca P 83db006ab6a44dcebcf71eac5e82f47d [term 1 FOLLOWER]: Leader election won for term 1
I20260619 10:19:36.081993 15396 raft_consensus.cc:686] T 8704c4b5976f419fa4e68e28ba8f3eca P 83db006ab6a44dcebcf71eac5e82f47d [term 1 LEADER]: Becoming Leader. State: Replica: 83db006ab6a44dcebcf71eac5e82f47d, State: Running, Role: LEADER
I20260619 10:19:36.082080 15396 consensus_queue.cc:237] T 8704c4b5976f419fa4e68e28ba8f3eca P 83db006ab6a44dcebcf71eac5e82f47d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "302e88a2fee04d3383b149616366acb0" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 43999 } } peers { permanent_uuid: "83db006ab6a44dcebcf71eac5e82f47d" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 38041 } } peers { permanent_uuid: "a392d075e14f4342808c63d2c776535d" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 39665 } }
I20260619 10:19:36.082820 15063 catalog_manager.cc:5495] T 8704c4b5976f419fa4e68e28ba8f3eca P 83db006ab6a44dcebcf71eac5e82f47d reported cstate change: term changed from 0 to 1, leader changed from <none> to 83db006ab6a44dcebcf71eac5e82f47d (127.0.206.66). New cstate: current_term: 1 leader_uuid: "83db006ab6a44dcebcf71eac5e82f47d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "302e88a2fee04d3383b149616366acb0" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 43999 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "83db006ab6a44dcebcf71eac5e82f47d" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 38041 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a392d075e14f4342808c63d2c776535d" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 39665 } health_report { overall_health: UNKNOWN } } }
I20260619 10:19:36.099314 15397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:19:36.097092 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47030 (local address 127.0.206.66:38041)
0619 10:19:36.097132 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0619 10:19:36.097134 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 10:19:36.097141 (+ 7us) server_negotiation.cc:407] Connection header received
0619 10:19:36.097222 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:19:36.097222 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:19:36.097238 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:19:36.097335 (+ 97us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 10:19:36.097415 (+ 80us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:19:36.097902 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:19:36.098560 (+ 658us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:19:36.098819 (+ 259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:19:36.098872 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:19:36.099025 (+ 153us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 10:19:36.099122 (+ 97us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 10:19:36.099174 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0619 10:19:36.099191 (+ 17us) server_negotiation.cc:299] Negotiation successful
0619 10:19:36.099209 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260619 10:19:36.100872 15206 raft_consensus.cc:1261] T 8704c4b5976f419fa4e68e28ba8f3eca P a392d075e14f4342808c63d2c776535d [term 1 FOLLOWER]: Refusing update from remote peer 83db006ab6a44dcebcf71eac5e82f47d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 10:19:36.100872 15481 raft_consensus.cc:1261] T 8704c4b5976f419fa4e68e28ba8f3eca P 302e88a2fee04d3383b149616366acb0 [term 1 FOLLOWER]: Refusing update from remote peer 83db006ab6a44dcebcf71eac5e82f47d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 10:19:36.101186 15396 consensus_queue.cc:1035] T 8704c4b5976f419fa4e68e28ba8f3eca P 83db006ab6a44dcebcf71eac5e82f47d [LEADER]: Connected to new peer: Peer: permanent_uuid: "302e88a2fee04d3383b149616366acb0" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 43999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 10:19:36.101341 15540 consensus_queue.cc:1035] T 8704c4b5976f419fa4e68e28ba8f3eca P 83db006ab6a44dcebcf71eac5e82f47d [LEADER]: Connected to new peer: Peer: permanent_uuid: "a392d075e14f4342808c63d2c776535d" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 39665 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 10:19:36.103652 15544 mvcc.cc:205] Tried to move back new op lower bound from 7298516484506402816 to 7298516484432740352. Current Snapshot: MvccSnapshot[applied={T|T < 7298516484506402816}]
I20260619 10:19:36.103976 15543 mvcc.cc:205] Tried to move back new op lower bound from 7298516484506402816 to 7298516484432740352. Current Snapshot: MvccSnapshot[applied={T|T < 7298516484506402816 or (T in {7298516484506402816})}]
I20260619 10:19:36.104094 15545 mvcc.cc:205] Tried to move back new op lower bound from 7298516484506402816 to 7298516484432740352. Current Snapshot: MvccSnapshot[applied={T|T < 7298516484506402816}]
I20260619 10:19:36.106154 15064 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:49956:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 10:19:36.106253 15064 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:49956:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 10:19:36.107008 15064 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 33de2c9a9d4148a281c87656d617ca65: Sending DeleteTablet for 3 replicas of tablet 8704c4b5976f419fa4e68e28ba8f3eca
I20260619 10:19:36.107331 15186 tablet_service.cc:1514] Processing DeleteTablet for tablet 8704c4b5976f419fa4e68e28ba8f3eca with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 10:19:36 UTC) from {username='kudu', principal='kudu/127.0.206.126@KRBTEST.COM'} at 127.0.0.1:57040
I20260619 10:19:36.107438 15261 tablet_replica.cc:330] T 8704c4b5976f419fa4e68e28ba8f3eca P a392d075e14f4342808c63d2c776535d: stopping tablet replica
I20260619 10:19:36.107535 15261 raft_consensus.cc:2229] T 8704c4b5976f419fa4e68e28ba8f3eca P a392d075e14f4342808c63d2c776535d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 10:19:36.107599 15261 raft_consensus.cc:2258] T 8704c4b5976f419fa4e68e28ba8f3eca P a392d075e14f4342808c63d2c776535d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 10:19:36.107945 15261 ts_tablet_manager.cc:1905] T 8704c4b5976f419fa4e68e28ba8f3eca P a392d075e14f4342808c63d2c776535d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 10:19:36.108314 15461 tablet_service.cc:1514] Processing DeleteTablet for tablet 8704c4b5976f419fa4e68e28ba8f3eca with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 10:19:36 UTC) from {username='kudu', principal='kudu/127.0.206.126@KRBTEST.COM'} at 127.0.0.1:56282
I20260619 10:19:36.108538 15553 tablet_replica.cc:330] T 8704c4b5976f419fa4e68e28ba8f3eca P 302e88a2fee04d3383b149616366acb0: stopping tablet replica
I20260619 10:19:36.108556 825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTU9OP6/build/release/bin/kudu with pid 15102
I20260619 10:19:36.108687 15553 raft_consensus.cc:2229] T 8704c4b5976f419fa4e68e28ba8f3eca P 302e88a2fee04d3383b149616366acb0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 10:19:36.108831 15553 raft_consensus.cc:2258] T 8704c4b5976f419fa4e68e28ba8f3eca P 302e88a2fee04d3383b149616366acb0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 10:19:36.109325 15553 ts_tablet_manager.cc:1905] T 8704c4b5976f419fa4e68e28ba8f3eca P 302e88a2fee04d3383b149616366acb0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 10:19:36.111284 15553 ts_tablet_manager.cc:1918] T 8704c4b5976f419fa4e68e28ba8f3eca P 302e88a2fee04d3383b149616366acb0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 10:19:36.111362 15325 tablet_service.cc:1514] Processing DeleteTablet for tablet 8704c4b5976f419fa4e68e28ba8f3eca with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 10:19:36 UTC) from {username='kudu', principal='kudu/127.0.206.126@KRBTEST.COM'} at 127.0.0.1:47022
I20260619 10:19:36.111451 15398 tablet_replica.cc:330] T 8704c4b5976f419fa4e68e28ba8f3eca P 83db006ab6a44dcebcf71eac5e82f47d: stopping tablet replica
I20260619 10:19:36.111402 15553 log.cc:1196] T 8704c4b5976f419fa4e68e28ba8f3eca P 302e88a2fee04d3383b149616366acb0: Deleting WAL directory at /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781864224577058-825-0/minicluster-data/ts-2/wal/wals/8704c4b5976f419fa4e68e28ba8f3eca
I20260619 10:19:36.111502 15398 raft_consensus.cc:2229] T 8704c4b5976f419fa4e68e28ba8f3eca P 83db006ab6a44dcebcf71eac5e82f47d [term 1 LEADER]: Raft consensus shutting down.
I20260619 10:19:36.111598 15398 raft_consensus.cc:2258] T 8704c4b5976f419fa4e68e28ba8f3eca P 83db006ab6a44dcebcf71eac5e82f47d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 10:19:36.111907 15553 ts_tablet_manager.cc:1939] T 8704c4b5976f419fa4e68e28ba8f3eca P 302e88a2fee04d3383b149616366acb0: Deleting consensus metadata
I20260619 10:19:36.112489 15049 catalog_manager.cc:4841] TS 302e88a2fee04d3383b149616366acb0 (127.0.206.67:43999): tablet 8704c4b5976f419fa4e68e28ba8f3eca (table test-table [id=8a5cff0b3fe54d189d233697bd096456]) successfully deleted
I20260619 10:19:36.112680 15398 ts_tablet_manager.cc:1905] T 8704c4b5976f419fa4e68e28ba8f3eca P 83db006ab6a44dcebcf71eac5e82f47d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 10:19:36.113770 15398 ts_tablet_manager.cc:1918] T 8704c4b5976f419fa4e68e28ba8f3eca P 83db006ab6a44dcebcf71eac5e82f47d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 10:19:36.113837 15398 log.cc:1196] T 8704c4b5976f419fa4e68e28ba8f3eca P 83db006ab6a44dcebcf71eac5e82f47d: Deleting WAL directory at /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781864224577058-825-0/minicluster-data/ts-1/wal/wals/8704c4b5976f419fa4e68e28ba8f3eca
I20260619 10:19:36.114066 15398 ts_tablet_manager.cc:1939] T 8704c4b5976f419fa4e68e28ba8f3eca P 83db006ab6a44dcebcf71eac5e82f47d: Deleting consensus metadata
I20260619 10:19:36.114423 15049 catalog_manager.cc:4841] TS 83db006ab6a44dcebcf71eac5e82f47d (127.0.206.66:38041): tablet 8704c4b5976f419fa4e68e28ba8f3eca (table test-table [id=8a5cff0b3fe54d189d233697bd096456]) successfully deleted
W20260619 10:19:36.115242 15050 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.0.206.65:39665 (error 108)
I20260619 10:19:36.115597 825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTU9OP6/build/release/bin/kudu with pid 15256
I20260619 10:19:36.115732 15259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:19:36.115588 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.65:39665 (local address 127.0.0.1:57052)
0619 10:19:36.115653 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0619 10:19:36.115695 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.65:39665: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260619 10:19:36.118779 15050 catalog_manager.cc:4584] TS a392d075e14f4342808c63d2c776535d (127.0.206.65:39665): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 8704c4b5976f419fa4e68e28ba8f3eca: Network error: Client connection negotiation failed: client connection to 127.0.206.65:39665: connect: Connection refused (error 111)
I20260619 10:19:36.121233 825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTU9OP6/build/release/bin/kudu with pid 15395
I20260619 10:19:36.125447 825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTU9OP6/build/release/bin/kudu with pid 15031
I20260619 10:19:36.130566 825 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTU9OP6/build/release/bin/kudu
/tmp/dist-test-taskTU9OP6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781864224577058-825-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781864224577058-825-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781864224577058-825-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781864224577058-825-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.206.126:36823
--webserver_interface=127.0.206.126
--webserver_port=36799
--keytab_file=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781864224577058-825-0/krb5kdc/kudu_127.0.206.126.keytab
--principal=kudu/127.0.206.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.206.84:42497
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.206.126:36823
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781864224577058-825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781864224577058-825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781864224577058-825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 10:19:36.213063 15555 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 10:19:36.213258 15555 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 10:19:36.213279 15555 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 10:19:36.214394 15555 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 10:19:36.214438 15555 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 10:19:36.214453 15555 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 10:19:36.214465 15555 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 10:19:36.214478 15555 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 10:19:36.215977 15555 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.206.84:42497
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781864224577058-825-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781864224577058-825-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.206.126:36823
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781864224577058-825-0/krb5kdc/kudu_127.0.206.126.keytab
--principal=kudu/127.0.206.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.206.126:36823
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781864224577058-825-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.206.126
--webserver_port=36799
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15555
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781864224577058-825-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 09:32:44 UTC on ccycloud.quasar-gbtldp.root.comops.site
build id 4455440
I20260619 10:19:36.216158 15555 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 10:19:36.216322 15555 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 10:19:36.220940 15563 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 10:19:36.220953 15560 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 10:19:36.220956 15561 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 10:19:36.220985 15555 server_base.cc:1034] running on GCE node
I20260619 10:19:36.221321 15555 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 10:19:36.221597 15555 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 10:19:36.222734 15555 hybrid_clock.cc:648] HybridClock initialized: now 1781864376222723 us; error 28 us; skew 500 ppm
Jun 19 10:19:36 dist-test-slave-6pt2 krb5kdc[14486](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864376, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 10:19:36.225028 15555 init.cc:377] Logged in from keytab as kudu/127.0.206.126@KRBTEST.COM (short username kudu)
I20260619 10:19:36.225476 15555 webserver.cc:460] Webserver started at http://127.0.206.126:36799/ using document root <none> and password file <none>
I20260619 10:19:36.225689 15555 fs_manager.cc:359] Metadata directory not provided
I20260619 10:19:36.225759 15555 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 10:19:36.227118 15555 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 10:19:36.227727 15570 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 10:19:36.227895 15555 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 10:19:36.227991 15555 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781864224577058-825-0/minicluster-data/master-0/data,/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781864224577058-825-0/minicluster-data/master-0/wal
uuid: "33de2c9a9d4148a281c87656d617ca65"
format_stamp: "Formatted at 2026-06-19 10:19:34 on dist-test-slave-6pt2"
server_key: "df2feca8e810e6795a30dfc48cdad65a"
server_key_iv: "fdff8c532fcc333feada82355159374c"
server_key_version: "encryptionkey@0"
I20260619 10:19:36.228334 15555 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781864224577058-825-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781864224577058-825-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781864224577058-825-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 10:19:36.246043 15555 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 10:19:36.247821 15555 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 10:19:36.247985 15555 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 10:19:36.252063 15555 rpc_server.cc:307] RPC server started. Bound to: 127.0.206.126:36823
I20260619 10:19:36.252096 15622 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.206.126:36823 every 8 connection(s)
I20260619 10:19:36.252455 15555 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781864224577058-825-0/minicluster-data/master-0/data/info.pb
I20260619 10:19:36.253870 15623 sys_catalog.cc:263] Verifying existing consensus state
I20260619 10:19:36.254509 15623 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 33de2c9a9d4148a281c87656d617ca65: Bootstrap starting.
I20260619 10:19:36.254995 825 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTU9OP6/build/release/bin/kudu as pid 15555
I20260619 10:19:36.255280 825 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTU9OP6/build/release/bin/kudu
/tmp/dist-test-taskTU9OP6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781864224577058-825-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781864224577058-825-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781864224577058-825-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781864224577058-825-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.206.65:39665
--local_ip_for_outbound_sockets=127.0.206.65
--tserver_master_addrs=127.0.206.126:36823
--webserver_port=38917
--webserver_interface=127.0.206.65
--keytab_file=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781864224577058-825-0/krb5kdc/kudu_127.0.206.65.keytab
--principal=kudu/127.0.206.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.206.84:42497
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781864224577058-825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781864224577058-825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781864224577058-825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 10:19:36.263562 15623 log.cc:824] T 00000000000000000000000000000000 P 33de2c9a9d4148a281c87656d617ca65: Log is configured to *not* fsync() on all Append() calls
I20260619 10:19:36.265640 15623 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 33de2c9a9d4148a281c87656d617ca65: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260619 10:19:36.265940 15623 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 33de2c9a9d4148a281c87656d617ca65: Bootstrap complete.
I20260619 10:19:36.268150 15623 raft_consensus.cc:348] T 00000000000000000000000000000000 P 33de2c9a9d4148a281c87656d617ca65 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "33de2c9a9d4148a281c87656d617ca65" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 36823 } }
I20260619 10:19:36.268361 15623 raft_consensus.cc:729] T 00000000000000000000000000000000 P 33de2c9a9d4148a281c87656d617ca65 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 33de2c9a9d4148a281c87656d617ca65, State: Initialized, Role: FOLLOWER
I20260619 10:19:36.268525 15623 consensus_queue.cc:260] T 00000000000000000000000000000000 P 33de2c9a9d4148a281c87656d617ca65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "33de2c9a9d4148a281c87656d617ca65" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 36823 } }
I20260619 10:19:36.268602 15623 raft_consensus.cc:388] T 00000000000000000000000000000000 P 33de2c9a9d4148a281c87656d617ca65 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 10:19:36.268653 15623 raft_consensus.cc:482] T 00000000000000000000000000000000 P 33de2c9a9d4148a281c87656d617ca65 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 10:19:36.268697 15623 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 33de2c9a9d4148a281c87656d617ca65 [term 2 FOLLOWER]: Advancing to term 3
I20260619 10:19:36.269661 15623 raft_consensus.cc:504] T 00000000000000000000000000000000 P 33de2c9a9d4148a281c87656d617ca65 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "33de2c9a9d4148a281c87656d617ca65" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 36823 } }
I20260619 10:19:36.269744 15623 leader_election.cc:302] T 00000000000000000000000000000000 P 33de2c9a9d4148a281c87656d617ca65 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 33de2c9a9d4148a281c87656d617ca65; no voters:
I20260619 10:19:36.269922 15623 leader_election.cc:288] T 00000000000000000000000000000000 P 33de2c9a9d4148a281c87656d617ca65 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260619 10:19:36.269956 15627 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 33de2c9a9d4148a281c87656d617ca65 [term 3 FOLLOWER]: Leader election won for term 3
I20260619 10:19:36.270064 15627 raft_consensus.cc:686] T 00000000000000000000000000000000 P 33de2c9a9d4148a281c87656d617ca65 [term 3 LEADER]: Becoming Leader. State: Replica: 33de2c9a9d4148a281c87656d617ca65, State: Running, Role: LEADER
I20260619 10:19:36.270146 15627 consensus_queue.cc:237] T 00000000000000000000000000000000 P 33de2c9a9d4148a281c87656d617ca65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "33de2c9a9d4148a281c87656d617ca65" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 36823 } }
I20260619 10:19:36.270180 15623 sys_catalog.cc:564] T 00000000000000000000000000000000 P 33de2c9a9d4148a281c87656d617ca65 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 10:19:36.270421 15627 sys_catalog.cc:455] T 00000000000000000000000000000000 P 33de2c9a9d4148a281c87656d617ca65 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 33de2c9a9d4148a281c87656d617ca65. Latest consensus state: current_term: 3 leader_uuid: "33de2c9a9d4148a281c87656d617ca65" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "33de2c9a9d4148a281c87656d617ca65" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 36823 } } }
I20260619 10:19:36.270509 15627 sys_catalog.cc:458] T 00000000000000000000000000000000 P 33de2c9a9d4148a281c87656d617ca65 [sys.catalog]: This master's current role is: LEADER
I20260619 10:19:36.270424 15628 sys_catalog.cc:455] T 00000000000000000000000000000000 P 33de2c9a9d4148a281c87656d617ca65 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "33de2c9a9d4148a281c87656d617ca65" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "33de2c9a9d4148a281c87656d617ca65" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 36823 } } }
I20260619 10:19:36.270680 15628 sys_catalog.cc:458] T 00000000000000000000000000000000 P 33de2c9a9d4148a281c87656d617ca65 [sys.catalog]: This master's current role is: LEADER
I20260619 10:19:36.270741 15632 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 10:19:36.271688 15632 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 10:19:36.271808 15632 catalog_manager.cc:1259] Loaded cluster ID: ffbf713c1b564873b61b865becdfe3dd
I20260619 10:19:36.272040 15632 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260619 10:19:36.272298 15632 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7fa90252d9ad google::LogMessage::Fail() at ??:0
@ 0x7fa90253088a google::LogMessage::SendToLog() at ??:0
@ 0x7fa90252d4de google::LogMessage::Flush() at ??:0
@ 0x7fa90252ea19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7fa90a44e0f1 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_.isra.591 at ??:0
@ 0x7fa90a4578b9 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7fa9043fb0d3 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7fa9043f0954 kudu::Thread::SuperviseThread() at ??:0
@ 0x7fa9052546db start_thread at ??:0
@ 0x7fa90102b61f clone at ??:0
W20260619 10:19:36.368975 15625 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 10:19:36.369225 15625 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 10:19:36.369258 15625 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 10:19:36.370915 15625 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 10:19:36.370970 15625 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 10:19:36.371007 15625 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.206.65
I20260619 10:19:36.373128 15625 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.206.84:42497
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781864224577058-825-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781864224577058-825-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781864224577058-825-0/krb5kdc/kudu_127.0.206.65.keytab
--principal=kudu/127.0.206.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.206.65:39665
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781864224577058-825-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.206.65
--webserver_port=38917
--tserver_master_addrs=127.0.206.126:36823
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15625
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.206.65
--log_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781864224577058-825-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 09:32:44 UTC on ccycloud.quasar-gbtldp.root.comops.site
build id 4455440
I20260619 10:19:36.373363 15625 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 10:19:36.373631 15625 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 10:19:36.378764 15660 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 10:19:36.378811 15625 server_base.cc:1034] running on GCE node
W20260619 10:19:36.378731 15659 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 10:19:36.378953 15662 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 10:19:36.379136 15625 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 10:19:36.379429 15625 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 10:19:36.380586 15625 hybrid_clock.cc:648] HybridClock initialized: now 1781864376380563 us; error 41 us; skew 500 ppm
Jun 19 10:19:36 dist-test-slave-6pt2 krb5kdc[14486](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864376, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 10:19:36.383095 15625 init.cc:377] Logged in from keytab as kudu/127.0.206.65@KRBTEST.COM (short username kudu)
I20260619 10:19:36.383571 15625 webserver.cc:460] Webserver started at http://127.0.206.65:38917/ using document root <none> and password file <none>
I20260619 10:19:36.383790 15625 fs_manager.cc:359] Metadata directory not provided
I20260619 10:19:36.383855 15625 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 10:19:36.385162 15625 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 10:19:36.385924 15669 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 10:19:36.386119 15625 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 10:19:36.386215 15625 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781864224577058-825-0/minicluster-data/ts-0/data,/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781864224577058-825-0/minicluster-data/ts-0/wal
uuid: "a392d075e14f4342808c63d2c776535d"
format_stamp: "Formatted at 2026-06-19 10:19:35 on dist-test-slave-6pt2"
server_key: "98abd49076517c583736948e0499b301"
server_key_iv: "dd592bbd5493ff87397663fbbc024616"
server_key_version: "encryptionkey@0"
I20260619 10:19:36.386545 15625 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781864224577058-825-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781864224577058-825-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781864224577058-825-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 10:19:36.396997 15625 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 10:19:36.399209 15625 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 10:19:36.399354 15625 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 10:19:36.399573 15625 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 10:19:36.400007 15676 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
W20260619 10:19:36.401809 15676 ts_tablet_manager.cc:1852] T 8704c4b5976f419fa4e68e28ba8f3eca P a392d075e14f4342808c63d2c776535d: Tablet Manager startup: Rolling forward tablet deletion of type TABLET_DATA_DELETED
I20260619 10:19:36.402154 15676 ts_tablet_manager.cc:1905] T 8704c4b5976f419fa4e68e28ba8f3eca P a392d075e14f4342808c63d2c776535d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 10:19:36.404618 15676 ts_tablet_manager.cc:1918] T 8704c4b5976f419fa4e68e28ba8f3eca P a392d075e14f4342808c63d2c776535d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 10:19:36.404716 15676 log.cc:1196] T 8704c4b5976f419fa4e68e28ba8f3eca P a392d075e14f4342808c63d2c776535d: Deleting WAL directory at /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781864224577058-825-0/minicluster-data/ts-0/wal/wals/8704c4b5976f419fa4e68e28ba8f3eca
I20260619 10:19:36.405023 15676 ts_tablet_manager.cc:1939] T 8704c4b5976f419fa4e68e28ba8f3eca P a392d075e14f4342808c63d2c776535d: Deleting consensus metadata
I20260619 10:19:36.405256 15625 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 0 live tablets)
I20260619 10:19:36.405315 15625 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.005s user 0.000s sys 0.000s
I20260619 10:19:36.405359 15625 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 10:19:36.405400 15625 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 10:19:36.412632 15625 rpc_server.cc:307] RPC server started. Bound to: 127.0.206.65:39665
I20260619 10:19:36.412707 15783 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.206.65:39665 every 8 connection(s)
I20260619 10:19:36.413019 15625 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781864224577058-825-0/minicluster-data/ts-0/data/info.pb
I20260619 10:19:36.420617 825 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTU9OP6/build/release/bin/kudu as pid 15625
I20260619 10:19:36.420900 825 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTU9OP6/build/release/bin/kudu
/tmp/dist-test-taskTU9OP6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781864224577058-825-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781864224577058-825-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781864224577058-825-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781864224577058-825-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.206.66:38041
--local_ip_for_outbound_sockets=127.0.206.66
--tserver_master_addrs=127.0.206.126:36823
--webserver_port=45373
--webserver_interface=127.0.206.66
--keytab_file=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781864224577058-825-0/krb5kdc/kudu_127.0.206.66.keytab
--principal=kudu/127.0.206.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.206.84:42497
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781864224577058-825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781864224577058-825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781864224577058-825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 10:19:36.467295 15786 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 10:19:36.413672 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:36823 (local address 127.0.206.65:59877)
0619 10:19:36.413848 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0619 10:19:36.413892 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0619 10:19:36.414362 (+ 470us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 10:19:36.467077 (+ 52715us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.126:36823: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":100,"threads_started":1}
W20260619 10:19:36.467577 15784 heartbeater.cc:644] Failed to heartbeat to 127.0.206.126:36823 (0 consecutive failures): Network error: Failed to ping master at 127.0.206.126:36823: Client connection negotiation failed: client connection to 127.0.206.126:36823: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260619 10:19:36.467947 15786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:19:36.467829 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:36823 (local address 127.0.206.65:54575)
0619 10:19:36.467884 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0619 10:19:36.467924 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.126:36823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260619 10:19:36.468286 15786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:19:36.468197 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:36823 (local address 127.0.206.65:35143)
0619 10:19:36.468240 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0619 10:19:36.468268 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.126:36823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260619 10:19:36.468433 15784 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260619 10:19:36.501900 15787 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 10:19:36.502084 15787 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 10:19:36.502110 15787 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 10:19:36.503311 15787 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 10:19:36.503351 15787 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 10:19:36.503404 15787 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.206.66
I20260619 10:19:36.504825 15787 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.206.84:42497
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781864224577058-825-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781864224577058-825-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781864224577058-825-0/krb5kdc/kudu_127.0.206.66.keytab
--principal=kudu/127.0.206.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.206.66:38041
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781864224577058-825-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.206.66
--webserver_port=45373
--tserver_master_addrs=127.0.206.126:36823
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15787
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.206.66
--log_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781864224577058-825-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 09:32:44 UTC on ccycloud.quasar-gbtldp.root.comops.site
build id 4455440
I20260619 10:19:36.505059 15787 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 10:19:36.505275 15787 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 10:19:36.509934 15795 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 10:19:36.509917 15792 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 10:19:36.509917 15793 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 10:19:36.510136 15787 server_base.cc:1034] running on GCE node
I20260619 10:19:36.510411 15787 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 10:19:36.510696 15787 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 10:19:36.511834 15787 hybrid_clock.cc:648] HybridClock initialized: now 1781864376511813 us; error 38 us; skew 500 ppm
Jun 19 10:19:36 dist-test-slave-6pt2 krb5kdc[14486](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864376, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 10:19:36.514202 15787 init.cc:377] Logged in from keytab as kudu/127.0.206.66@KRBTEST.COM (short username kudu)
I20260619 10:19:36.514714 15787 webserver.cc:460] Webserver started at http://127.0.206.66:45373/ using document root <none> and password file <none>
I20260619 10:19:36.514947 15787 fs_manager.cc:359] Metadata directory not provided
I20260619 10:19:36.515012 15787 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 10:19:36.516276 15787 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 10:19:36.516791 15802 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 10:19:36.516944 15787 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 10:19:36.517055 15787 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781864224577058-825-0/minicluster-data/ts-1/data,/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781864224577058-825-0/minicluster-data/ts-1/wal
uuid: "83db006ab6a44dcebcf71eac5e82f47d"
format_stamp: "Formatted at 2026-06-19 10:19:35 on dist-test-slave-6pt2"
server_key: "34291ad4f48d2234c2443a0a8ef84bce"
server_key_iv: "d28f26ab00c975a1d8d7da5b6d8c0e31"
server_key_version: "encryptionkey@0"
I20260619 10:19:36.517373 15787 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781864224577058-825-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781864224577058-825-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781864224577058-825-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 10:19:36.530498 15787 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 10:19:36.532224 15787 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 10:19:36.532385 15787 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 10:19:36.532598 15787 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 10:19:36.532886 15787 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 10:19:36.532940 15787 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 10:19:36.532980 15787 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 10:19:36.533010 15787 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 10:19:36.538717 15787 rpc_server.cc:307] RPC server started. Bound to: 127.0.206.66:38041
I20260619 10:19:36.538769 15915 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.206.66:38041 every 8 connection(s)
I20260619 10:19:36.539073 15787 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781864224577058-825-0/minicluster-data/ts-1/data/info.pb
I20260619 10:19:36.540009 15918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:19:36.539673 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:36823 (local address 127.0.206.66:47099)
0619 10:19:36.539817 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0619 10:19:36.539881 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.126:36823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":70,"threads_started":1}
W20260619 10:19:36.540385 15916 heartbeater.cc:644] Failed to heartbeat to 127.0.206.126:36823 (0 consecutive failures): Network error: Failed to ping master at 127.0.206.126:36823: Client connection negotiation failed: client connection to 127.0.206.126:36823: connect: Connection refused (error 111)
I20260619 10:19:36.540628 15918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:19:36.540543 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:36823 (local address 127.0.206.66:53291)
0619 10:19:36.540587 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0619 10:19:36.540612 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.126:36823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
I20260619 10:19:36.540880 15918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:19:36.540830 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:36823 (local address 127.0.206.66:40477)
0619 10:19:36.540850 (+ 20us) negotiation.cc:107] Waiting for socket to connect
0619 10:19:36.540868 (+ 18us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.126:36823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":12}
W20260619 10:19:36.540983 15916 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260619 10:19:36.545703 825 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTU9OP6/build/release/bin/kudu as pid 15787
I20260619 10:19:36.545941 825 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTU9OP6/build/release/bin/kudu
/tmp/dist-test-taskTU9OP6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781864224577058-825-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781864224577058-825-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781864224577058-825-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781864224577058-825-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.206.67:43999
--local_ip_for_outbound_sockets=127.0.206.67
--tserver_master_addrs=127.0.206.126:36823
--webserver_port=33311
--webserver_interface=127.0.206.67
--keytab_file=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781864224577058-825-0/krb5kdc/kudu_127.0.206.67.keytab
--principal=kudu/127.0.206.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.206.84:42497
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781864224577058-825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781864224577058-825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781864224577058-825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 10:19:36.626744 15919 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 10:19:36.626920 15919 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 10:19:36.626943 15919 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 10:19:36.628005 15919 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 10:19:36.628038 15919 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 10:19:36.628064 15919 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.206.67
I20260619 10:19:36.629406 15919 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.206.84:42497
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781864224577058-825-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781864224577058-825-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781864224577058-825-0/krb5kdc/kudu_127.0.206.67.keytab
--principal=kudu/127.0.206.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.206.67:43999
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781864224577058-825-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.206.67
--webserver_port=33311
--tserver_master_addrs=127.0.206.126:36823
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15919
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.206.67
--log_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781864224577058-825-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 09:32:44 UTC on ccycloud.quasar-gbtldp.root.comops.site
build id 4455440
I20260619 10:19:36.629570 15919 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 10:19:36.629727 15919 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 10:19:36.634516 15924 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 10:19:36.634507 15927 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 10:19:36.634768 15919 server_base.cc:1034] running on GCE node
W20260619 10:19:36.634526 15925 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 10:19:36.635001 15919 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 10:19:36.635229 15919 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 10:19:36.636382 15919 hybrid_clock.cc:648] HybridClock initialized: now 1781864376636371 us; error 27 us; skew 500 ppm
Jun 19 10:19:36 dist-test-slave-6pt2 krb5kdc[14486](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864376, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 10:19:36.638835 15919 init.cc:377] Logged in from keytab as kudu/127.0.206.67@KRBTEST.COM (short username kudu)
I20260619 10:19:36.639263 15919 webserver.cc:460] Webserver started at http://127.0.206.67:33311/ using document root <none> and password file <none>
I20260619 10:19:36.639452 15919 fs_manager.cc:359] Metadata directory not provided
I20260619 10:19:36.639492 15919 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 10:19:36.640676 15919 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260619 10:19:36.641224 15934 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 10:19:36.641415 15919 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260619 10:19:36.641500 15919 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781864224577058-825-0/minicluster-data/ts-2/data,/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781864224577058-825-0/minicluster-data/ts-2/wal
uuid: "302e88a2fee04d3383b149616366acb0"
format_stamp: "Formatted at 2026-06-19 10:19:35 on dist-test-slave-6pt2"
server_key: "1ad1ae9fd44e12b3d5b32f86927da910"
server_key_iv: "61906ee126d7435d5887f96c7259397f"
server_key_version: "encryptionkey@0"
I20260619 10:19:36.641798 15919 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781864224577058-825-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781864224577058-825-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781864224577058-825-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 10:19:36.652559 15919 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 10:19:36.654762 15919 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 10:19:36.654874 15919 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 10:19:36.655052 15919 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 10:19:36.655282 15919 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 10:19:36.655309 15919 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 10:19:36.655330 15919 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 10:19:36.655342 15919 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 10:19:36.660863 15919 rpc_server.cc:307] RPC server started. Bound to: 127.0.206.67:43999
I20260619 10:19:36.660933 16047 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.206.67:43999 every 8 connection(s)
I20260619 10:19:36.661252 15919 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781864224577058-825-0/minicluster-data/ts-2/data/info.pb
I20260619 10:19:36.662206 16050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:19:36.661866 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:36823 (local address 127.0.206.67:55821)
0619 10:19:36.662008 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0619 10:19:36.662071 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.126:36823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":71,"threads_started":1}
W20260619 10:19:36.662631 16048 heartbeater.cc:644] Failed to heartbeat to 127.0.206.126:36823 (0 consecutive failures): Network error: Failed to ping master at 127.0.206.126:36823: Client connection negotiation failed: client connection to 127.0.206.126:36823: connect: Connection refused (error 111)
I20260619 10:19:36.662932 16050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:19:36.662842 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:36823 (local address 127.0.206.67:43201)
0619 10:19:36.662888 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0619 10:19:36.662911 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.126:36823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
I20260619 10:19:36.663223 16050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:19:36.663139 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:36823 (local address 127.0.206.67:39883)
0619 10:19:36.663182 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0619 10:19:36.663203 (+ 21us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.126:36823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260619 10:19:36.663324 16048 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260619 10:19:36.670521 825 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTU9OP6/build/release/bin/kudu as pid 15919
I20260619 10:19:37.468976 16052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:19:37.468762 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:36823 (local address 127.0.206.65:34845)
0619 10:19:37.468882 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0619 10:19:37.468917 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.126:36823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":58,"threads_started":1}
I20260619 10:19:37.541428 16053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:19:37.541245 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:36823 (local address 127.0.206.66:39269)
0619 10:19:37.541352 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0619 10:19:37.541381 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.126:36823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":94,"thread_start_us":54,"threads_started":1}
I20260619 10:19:37.663913 16054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:19:37.663611 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:36823 (local address 127.0.206.67:45513)
0619 10:19:37.663828 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0619 10:19:37.663870 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.126:36823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":200,"thread_start_us":55,"threads_started":1}
I20260619 10:19:38.469708 16055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:19:38.469473 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:36823 (local address 127.0.206.65:48295)
0619 10:19:38.469599 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0619 10:19:38.469643 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.126:36823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":77,"threads_started":1}
I20260619 10:19:38.542073 16056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:19:38.541867 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:36823 (local address 127.0.206.66:50119)
0619 10:19:38.542000 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0619 10:19:38.542035 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.126:36823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":62,"threads_started":1}
I20260619 10:19:38.664595 16057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:19:38.664400 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:36823 (local address 127.0.206.67:41697)
0619 10:19:38.664527 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0619 10:19:38.664559 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.126:36823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":72,"threads_started":1}
I20260619 10:19:39.470475 16058 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:19:39.470262 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:36823 (local address 127.0.206.65:55591)
0619 10:19:39.470396 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0619 10:19:39.470426 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.126:36823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":76,"threads_started":1}
I20260619 10:19:39.542738 16059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:19:39.542503 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:36823 (local address 127.0.206.66:39359)
0619 10:19:39.542677 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0619 10:19:39.542706 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.126:36823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":160,"thread_start_us":71,"threads_started":1}
I20260619 10:19:39.665166 16060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:19:39.664972 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:36823 (local address 127.0.206.67:49063)
0619 10:19:39.665096 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0619 10:19:39.665132 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.126:36823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":73,"threads_started":1}
I20260619 10:19:40.471238 16061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:19:40.470992 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:36823 (local address 127.0.206.65:34383)
0619 10:19:40.471151 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0619 10:19:40.471191 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.126:36823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":82,"threads_started":1}
I20260619 10:19:40.543298 16062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:19:40.543103 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:36823 (local address 127.0.206.66:47201)
0619 10:19:40.543228 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0619 10:19:40.543257 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.126:36823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":74,"threads_started":1}
I20260619 10:19:40.665849 16063 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:19:40.665623 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:36823 (local address 127.0.206.67:59587)
0619 10:19:40.665749 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0619 10:19:40.665805 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.126:36823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":69,"threads_started":1}
I20260619 10:19:41.471846 16064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:19:41.471639 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:36823 (local address 127.0.206.65:57815)
0619 10:19:41.471760 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0619 10:19:41.471799 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.126:36823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":61,"threads_started":1}
I20260619 10:19:41.544000 16065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:19:41.543784 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:36823 (local address 127.0.206.66:35571)
0619 10:19:41.543920 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0619 10:19:41.543964 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.126:36823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":53,"threads_started":1}
I20260619 10:19:41.666512 16066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:19:41.666308 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:36823 (local address 127.0.206.67:47479)
0619 10:19:41.666433 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0619 10:19:41.666476 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.126:36823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":60,"threads_started":1}
I20260619 10:19:42.472553 16067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:19:42.472345 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:36823 (local address 127.0.206.65:49521)
0619 10:19:42.472471 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0619 10:19:42.472506 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.126:36823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":75,"threads_started":1}
I20260619 10:19:42.544611 16068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:19:42.544390 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:36823 (local address 127.0.206.66:35835)
0619 10:19:42.544540 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0619 10:19:42.544574 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.126:36823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":74,"threads_started":1}
I20260619 10:19:42.667184 16069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:19:42.666985 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:36823 (local address 127.0.206.67:50047)
0619 10:19:42.667112 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0619 10:19:42.667144 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.126:36823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":58,"threads_started":1}
I20260619 10:19:43.473218 16070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:19:43.473006 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:36823 (local address 127.0.206.65:55995)
0619 10:19:43.473129 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0619 10:19:43.473169 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.126:36823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":63,"threads_started":1}
I20260619 10:19:43.545338 16071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:19:43.545066 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:36823 (local address 127.0.206.66:40101)
0619 10:19:43.545241 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0619 10:19:43.545288 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.126:36823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":84,"threads_started":1}
I20260619 10:19:43.667809 16072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:19:43.667595 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:36823 (local address 127.0.206.67:45833)
0619 10:19:43.667742 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0619 10:19:43.667769 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.126:36823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":94,"threads_started":1}
I20260619 10:19:44.473902 16073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:19:44.473687 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:36823 (local address 127.0.206.65:57647)
0619 10:19:44.473824 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0619 10:19:44.473857 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.126:36823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":87,"threads_started":1}
I20260619 10:19:44.545944 16074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:19:44.545738 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:36823 (local address 127.0.206.66:40435)
0619 10:19:44.545871 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0619 10:19:44.545904 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.126:36823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":74,"threads_started":1}
I20260619 10:19:44.668434 16075 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:19:44.668218 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:36823 (local address 127.0.206.67:48765)
0619 10:19:44.668357 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0619 10:19:44.668394 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.126:36823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":56,"threads_started":1}
I20260619 10:19:45.474563 16076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:19:45.474366 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:36823 (local address 127.0.206.65:34473)
0619 10:19:45.474487 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0619 10:19:45.474518 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.126:36823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":76,"threads_started":1}
I20260619 10:19:45.546564 16077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:19:45.546361 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:36823 (local address 127.0.206.66:45083)
0619 10:19:45.546482 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0619 10:19:45.546519 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.126:36823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":66,"threads_started":1}
I20260619 10:19:45.669118 16078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:19:45.668929 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:36823 (local address 127.0.206.67:57739)
0619 10:19:45.669046 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0619 10:19:45.669079 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.126:36823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":59,"threads_started":1}
I20260619 10:19:46.475195 16079 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:19:46.474967 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:36823 (local address 127.0.206.65:49365)
0619 10:19:46.475111 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0619 10:19:46.475149 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.126:36823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":59,"threads_started":1}
I20260619 10:19:46.547190 16080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:19:46.546990 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:36823 (local address 127.0.206.66:40043)
0619 10:19:46.547107 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0619 10:19:46.547150 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.126:36823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":57,"threads_started":1}
I20260619 10:19:46.669746 16081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:19:46.669537 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:36823 (local address 127.0.206.67:52015)
0619 10:19:46.669665 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0619 10:19:46.669701 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.126:36823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":57,"threads_started":1}
I20260619 10:19:47.475945 16082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:19:47.475747 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:36823 (local address 127.0.206.65:43825)
0619 10:19:47.475864 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0619 10:19:47.475900 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.126:36823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":57,"threads_started":1}
I20260619 10:19:47.547761 16083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:19:47.547571 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:36823 (local address 127.0.206.66:41601)
0619 10:19:47.547689 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0619 10:19:47.547722 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.126:36823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":60,"threads_started":1}
I20260619 10:19:47.670414 16084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:19:47.670230 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:36823 (local address 127.0.206.67:33469)
0619 10:19:47.670343 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0619 10:19:47.670377 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.126:36823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":99,"thread_start_us":54,"threads_started":1}
I20260619 10:19:48.476676 16085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:19:48.476467 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:36823 (local address 127.0.206.65:41031)
0619 10:19:48.476593 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0619 10:19:48.476629 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.126:36823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":85,"threads_started":1}
I20260619 10:19:48.548306 16086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:19:48.548135 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:36823 (local address 127.0.206.66:45049)
0619 10:19:48.548240 (+ 105us) negotiation.cc:107] Waiting for socket to connect
0619 10:19:48.548269 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.126:36823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":91,"thread_start_us":56,"threads_started":1}
I20260619 10:19:48.671167 16087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:19:48.670966 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:36823 (local address 127.0.206.67:39505)
0619 10:19:48.671081 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0619 10:19:48.671114 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.126:36823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":58,"threads_started":1}
I20260619 10:19:49.477346 16088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:19:49.477147 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:36823 (local address 127.0.206.65:33289)
0619 10:19:49.477269 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0619 10:19:49.477304 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.126:36823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":53,"threads_started":1}
I20260619 10:19:49.548914 16089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:19:49.548722 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:36823 (local address 127.0.206.66:35819)
0619 10:19:49.548847 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0619 10:19:49.548877 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.126:36823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":76,"threads_started":1}
I20260619 10:19:49.671731 16090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:19:49.671555 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:36823 (local address 127.0.206.67:35659)
0619 10:19:49.671659 (+ 104us) negotiation.cc:107] Waiting for socket to connect
0619 10:19:49.671691 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.126:36823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":90,"thread_start_us":54,"threads_started":1}
I20260619 10:19:50.477934 16091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:19:50.477741 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:36823 (local address 127.0.206.65:37767)
0619 10:19:50.477856 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0619 10:19:50.477889 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.126:36823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":64,"threads_started":1}
I20260619 10:19:50.549458 16092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:19:50.549276 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:36823 (local address 127.0.206.66:46399)
0619 10:19:50.549383 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0619 10:19:50.549416 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.126:36823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":92,"thread_start_us":57,"threads_started":1}
I20260619 10:19:50.672381 16093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:19:50.672198 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:36823 (local address 127.0.206.67:57829)
0619 10:19:50.672314 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0619 10:19:50.672345 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.126:36823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":72,"threads_started":1}
I20260619 10:19:51.478577 16094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:19:51.478385 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:36823 (local address 127.0.206.65:33977)
0619 10:19:51.478499 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0619 10:19:51.478532 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.126:36823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":56,"threads_started":1}
I20260619 10:19:51.550218 16095 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:19:51.549926 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:36823 (local address 127.0.206.66:47295)
0619 10:19:51.550097 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0619 10:19:51.550129 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.126:36823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":104,"threads_started":1}
I20260619 10:19:51.673141 16096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:19:51.672945 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:36823 (local address 127.0.206.67:48719)
0619 10:19:51.673069 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0619 10:19:51.673103 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.126:36823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":65,"threads_started":1}
I20260619 10:19:51.675431 825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTU9OP6/build/release/bin/kudu with pid 15625
I20260619 10:19:51.679802 825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTU9OP6/build/release/bin/kudu with pid 15787
I20260619 10:19:51.683724 825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTU9OP6/build/release/bin/kudu with pid 15919
2026-06-19T10:19:51Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (18104 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781864224577058-825-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 10:19:51 dist-test-slave-6pt2 krb5kdc[16099](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 10:19:51 dist-test-slave-6pt2 krb5kdc[16099](info): set up 2 sockets
Jun 19 10:19:51 dist-test-slave-6pt2 krb5kdc[16099](info): commencing operation
krb5kdc: starting...
W20260619 10:19:52.714573 825 mini_kdc.cc:121] Time spent starting KDC: real 1.011s user 0.003s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781864224577058-825-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781864224577058-825-0/krb5kdc/test-user.keytab.
Jun 19 10:19:52 dist-test-slave-6pt2 krb5kdc[16099](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864392, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T10:19:52Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T10:19:52Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.206.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.206.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.206.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781864224577058-825-0/krb5kdc/kudu_127.0.206.126.keytab.
Entry for principal kudu/127.0.206.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781864224577058-825-0/krb5kdc/kudu_127.0.206.126.keytab.
I20260619 10:19:52.820947 825 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTU9OP6/build/release/bin/kudu
/tmp/dist-test-taskTU9OP6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781864224577058-825-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781864224577058-825-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781864224577058-825-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781864224577058-825-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.206.126:33511
--webserver_interface=127.0.206.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781864224577058-825-0/krb5kdc/kudu_127.0.206.126.keytab
--principal=kudu/127.0.206.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.206.84:43867
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.206.126:33511
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781864224577058-825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781864224577058-825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781864224577058-825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 10:19:52.898398 16112 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 10:19:52.898614 16112 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 10:19:52.898669 16112 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 10:19:52.899762 16112 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260619 10:19:52.899832 16112 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260619 10:19:52.899852 16112 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260619 10:19:52.899880 16112 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 10:19:52.899924 16112 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 10:19:52.899951 16112 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 10:19:52.899979 16112 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 10:19:52.900002 16112 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 10:19:52.901424 16112 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.206.84:43867
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781864224577058-825-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781864224577058-825-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.0.206.126:33511
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781864224577058-825-0/krb5kdc/kudu_127.0.206.126.keytab
--principal=kudu/127.0.206.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.206.126:33511
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781864224577058-825-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.206.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16112
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781864224577058-825-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 09:32:44 UTC on ccycloud.quasar-gbtldp.root.comops.site
build id 4455440
I20260619 10:19:52.901672 16112 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 10:19:52.901875 16112 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 10:19:52.906903 16118 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 10:19:52.906967 16112 server_base.cc:1034] running on GCE node
W20260619 10:19:52.906903 16117 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 10:19:52.906929 16120 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 10:19:52.907265 16112 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 10:19:52.907550 16112 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 10:19:52.908708 16112 hybrid_clock.cc:648] HybridClock initialized: now 1781864392908699 us; error 26 us; skew 500 ppm
Jun 19 10:19:52 dist-test-slave-6pt2 krb5kdc[16099](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864392, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 10:19:52.911176 16112 init.cc:377] Logged in from keytab as kudu/127.0.206.126@KRBTEST.COM (short username kudu)
I20260619 10:19:52.911520 16112 webserver.cc:460] Webserver started at http://127.0.206.126:43223/ using document root <none> and password file <none>
I20260619 10:19:52.911706 16112 fs_manager.cc:359] Metadata directory not provided
I20260619 10:19:52.911746 16112 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 10:19:52.911828 16112 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 10:19:52.912741 16112 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781864224577058-825-0/minicluster-data/master-0/data/instance:
uuid: "b4330665f37e4f6693f0900bed7dcfe6"
format_stamp: "Formatted at 2026-06-19 10:19:52 on dist-test-slave-6pt2"
server_key: "1b36e4882d0518269926abeae9ba3acc"
server_key_iv: "8dd3ce24ed149b1d6aab6338fcf9b6df"
server_key_version: "encryptionkey@0"
I20260619 10:19:52.913038 16112 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781864224577058-825-0/minicluster-data/master-0/wal/instance:
uuid: "b4330665f37e4f6693f0900bed7dcfe6"
format_stamp: "Formatted at 2026-06-19 10:19:52 on dist-test-slave-6pt2"
server_key: "1b36e4882d0518269926abeae9ba3acc"
server_key_iv: "8dd3ce24ed149b1d6aab6338fcf9b6df"
server_key_version: "encryptionkey@0"
I20260619 10:19:52.914207 16112 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260619 10:19:52.914860 16127 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 10:19:52.915004 16112 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260619 10:19:52.915079 16112 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781864224577058-825-0/minicluster-data/master-0/data,/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781864224577058-825-0/minicluster-data/master-0/wal
uuid: "b4330665f37e4f6693f0900bed7dcfe6"
format_stamp: "Formatted at 2026-06-19 10:19:52 on dist-test-slave-6pt2"
server_key: "1b36e4882d0518269926abeae9ba3acc"
server_key_iv: "8dd3ce24ed149b1d6aab6338fcf9b6df"
server_key_version: "encryptionkey@0"
I20260619 10:19:52.915156 16112 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781864224577058-825-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781864224577058-825-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781864224577058-825-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 10:19:52.932405 16112 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 10:19:52.934149 16112 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 10:19:52.934259 16112 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 10:19:52.937860 16112 rpc_server.cc:307] RPC server started. Bound to: 127.0.206.126:33511
I20260619 10:19:52.937911 16179 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.206.126:33511 every 8 connection(s)
I20260619 10:19:52.938235 16112 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781864224577058-825-0/minicluster-data/master-0/data/info.pb
I20260619 10:19:52.939131 16180 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 10:19:52.941335 16180 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b4330665f37e4f6693f0900bed7dcfe6: Bootstrap starting.
I20260619 10:19:52.941941 16180 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b4330665f37e4f6693f0900bed7dcfe6: Neither blocks nor log segments found. Creating new log.
I20260619 10:19:52.942224 16180 log.cc:824] T 00000000000000000000000000000000 P b4330665f37e4f6693f0900bed7dcfe6: Log is configured to *not* fsync() on all Append() calls
I20260619 10:19:52.942883 16180 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b4330665f37e4f6693f0900bed7dcfe6: No bootstrap required, opened a new log
I20260619 10:19:52.944429 16180 raft_consensus.cc:348] T 00000000000000000000000000000000 P b4330665f37e4f6693f0900bed7dcfe6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b4330665f37e4f6693f0900bed7dcfe6" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 33511 } }
I20260619 10:19:52.944574 16180 raft_consensus.cc:374] T 00000000000000000000000000000000 P b4330665f37e4f6693f0900bed7dcfe6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 10:19:52.944617 16180 raft_consensus.cc:729] T 00000000000000000000000000000000 P b4330665f37e4f6693f0900bed7dcfe6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b4330665f37e4f6693f0900bed7dcfe6, State: Initialized, Role: FOLLOWER
I20260619 10:19:52.944765 16180 consensus_queue.cc:260] T 00000000000000000000000000000000 P b4330665f37e4f6693f0900bed7dcfe6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b4330665f37e4f6693f0900bed7dcfe6" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 33511 } }
I20260619 10:19:52.944854 16180 raft_consensus.cc:388] T 00000000000000000000000000000000 P b4330665f37e4f6693f0900bed7dcfe6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 10:19:52.944895 16180 raft_consensus.cc:482] T 00000000000000000000000000000000 P b4330665f37e4f6693f0900bed7dcfe6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 10:19:52.944941 16180 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b4330665f37e4f6693f0900bed7dcfe6 [term 0 FOLLOWER]: Advancing to term 1
I20260619 10:19:52.945420 16180 raft_consensus.cc:504] T 00000000000000000000000000000000 P b4330665f37e4f6693f0900bed7dcfe6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b4330665f37e4f6693f0900bed7dcfe6" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 33511 } }
I20260619 10:19:52.945524 16180 leader_election.cc:302] T 00000000000000000000000000000000 P b4330665f37e4f6693f0900bed7dcfe6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b4330665f37e4f6693f0900bed7dcfe6; no voters:
I20260619 10:19:52.945726 16180 leader_election.cc:288] T 00000000000000000000000000000000 P b4330665f37e4f6693f0900bed7dcfe6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 10:19:52.945748 16183 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b4330665f37e4f6693f0900bed7dcfe6 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 10:19:52.945744 825 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTU9OP6/build/release/bin/kudu as pid 16112
I20260619 10:19:52.945888 825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781864224577058-825-0/minicluster-data/master-0/wal/instance
I20260619 10:19:52.945912 16183 raft_consensus.cc:686] T 00000000000000000000000000000000 P b4330665f37e4f6693f0900bed7dcfe6 [term 1 LEADER]: Becoming Leader. State: Replica: b4330665f37e4f6693f0900bed7dcfe6, State: Running, Role: LEADER
I20260619 10:19:52.946007 16180 sys_catalog.cc:564] T 00000000000000000000000000000000 P b4330665f37e4f6693f0900bed7dcfe6 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 10:19:52.946120 825 external_mini_cluster.cc:1428] Setting key 311ccea2072f320cb30c81c0c39010e6
I20260619 10:19:52.945982 16183 consensus_queue.cc:237] T 00000000000000000000000000000000 P b4330665f37e4f6693f0900bed7dcfe6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b4330665f37e4f6693f0900bed7dcfe6" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 33511 } }
I20260619 10:19:52.946555 16183 sys_catalog.cc:455] T 00000000000000000000000000000000 P b4330665f37e4f6693f0900bed7dcfe6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b4330665f37e4f6693f0900bed7dcfe6. Latest consensus state: current_term: 1 leader_uuid: "b4330665f37e4f6693f0900bed7dcfe6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b4330665f37e4f6693f0900bed7dcfe6" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 33511 } } }
I20260619 10:19:52.946729 16183 sys_catalog.cc:458] T 00000000000000000000000000000000 P b4330665f37e4f6693f0900bed7dcfe6 [sys.catalog]: This master's current role is: LEADER
I20260619 10:19:52.946792 16185 sys_catalog.cc:455] T 00000000000000000000000000000000 P b4330665f37e4f6693f0900bed7dcfe6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b4330665f37e4f6693f0900bed7dcfe6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b4330665f37e4f6693f0900bed7dcfe6" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 33511 } } }
I20260619 10:19:52.946908 16185 sys_catalog.cc:458] T 00000000000000000000000000000000 P b4330665f37e4f6693f0900bed7dcfe6 [sys.catalog]: This master's current role is: LEADER
I20260619 10:19:52.946996 16194 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 10:19:52.947540 16194 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 10:19:52.949846 16194 catalog_manager.cc:1347] Generated new cluster ID: a2afe9084c3b435aaa7c67b9feec0afd
I20260619 10:19:52.949914 16194 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Jun 19 10:19:52 dist-test-slave-6pt2 krb5kdc[16099](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864392, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.206.126@KRBTEST.COM
I20260619 10:19:52.957937 16191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:19:52.946561 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33310 (local address 127.0.206.126:33511)
0619 10:19:52.947054 (+ 493us) server_negotiation.cc:206] Beginning negotiation
0619 10:19:52.947063 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 10:19:52.947328 (+ 265us) server_negotiation.cc:407] Connection header received
0619 10:19:52.947741 (+ 413us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:19:52.947749 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:19:52.947805 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:19:52.947938 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:19:52.948765 (+ 827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:19:52.949740 (+ 975us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:19:52.950820 (+ 1080us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:19:52.951397 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:19:52.951492 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:19:52.954154 (+ 2662us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 10:19:52.954158 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 10:19:52.954158 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 10:19:52.954170 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0619 10:19:52.956172 (+ 2002us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:19:52.956540 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:19:52.956543 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:19:52.956544 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:19:52.956676 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:19:52.956893 (+ 217us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:19:52.956894 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:19:52.956894 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:19:52.957218 (+ 324us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 10:19:52.957371 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0619 10:19:52.957467 (+ 96us) server_negotiation.cc:299] Negotiation successful
0619 10:19:52.957721 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":442,"thread_start_us":103,"threads_started":1}
I20260619 10:19:52.980280 16194 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 10:19:52.980823 16194 catalog_manager.cc:1504] Loading token signing keys...
I20260619 10:19:52.986063 16194 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b4330665f37e4f6693f0900bed7dcfe6: Generated new TSK 0
I20260619 10:19:52.986266 16194 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.0.206.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.206.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.206.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781864224577058-825-0/krb5kdc/kudu_127.0.206.65.keytab.
Entry for principal kudu/127.0.206.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781864224577058-825-0/krb5kdc/kudu_127.0.206.65.keytab.
I20260619 10:19:53.013413 825 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTU9OP6/build/release/bin/kudu
/tmp/dist-test-taskTU9OP6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781864224577058-825-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781864224577058-825-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781864224577058-825-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781864224577058-825-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.206.65:0
--local_ip_for_outbound_sockets=127.0.206.65
--webserver_interface=127.0.206.65
--webserver_port=0
--tserver_master_addrs=127.0.206.126:33511
--keytab_file=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781864224577058-825-0/krb5kdc/kudu_127.0.206.65.keytab
--principal=kudu/127.0.206.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.206.84:43867
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781864224577058-825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781864224577058-825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781864224577058-825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 10:19:53.092723 16206 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 10:19:53.092928 16206 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 10:19:53.092952 16206 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 10:19:53.094028 16206 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 10:19:53.094062 16206 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 10:19:53.094087 16206 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.206.65
I20260619 10:19:53.095518 16206 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.206.84:43867
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781864224577058-825-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781864224577058-825-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781864224577058-825-0/krb5kdc/kudu_127.0.206.65.keytab
--principal=kudu/127.0.206.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.206.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781864224577058-825-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.206.65
--webserver_port=0
--tserver_master_addrs=127.0.206.126:33511
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16206
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.206.65
--log_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781864224577058-825-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 09:32:44 UTC on ccycloud.quasar-gbtldp.root.comops.site
build id 4455440
I20260619 10:19:53.095695 16206 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 10:19:53.095862 16206 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 10:19:53.100883 16211 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 10:19:53.100891 16214 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 10:19:53.100910 16212 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 10:19:53.100937 16206 server_base.cc:1034] running on GCE node
I20260619 10:19:53.101279 16206 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 10:19:53.101513 16206 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 10:19:53.102658 16206 hybrid_clock.cc:648] HybridClock initialized: now 1781864393102636 us; error 35 us; skew 500 ppm
Jun 19 10:19:53 dist-test-slave-6pt2 krb5kdc[16099](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864393, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 10:19:53.105147 16206 init.cc:377] Logged in from keytab as kudu/127.0.206.65@KRBTEST.COM (short username kudu)
I20260619 10:19:53.105576 16206 webserver.cc:460] Webserver started at http://127.0.206.65:38183/ using document root <none> and password file <none>
I20260619 10:19:53.105765 16206 fs_manager.cc:359] Metadata directory not provided
I20260619 10:19:53.105804 16206 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 10:19:53.105888 16206 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 10:19:53.106902 16206 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781864224577058-825-0/minicluster-data/ts-0/data/instance:
uuid: "d835bbc06b3d47f7b10e69f31a298ed9"
format_stamp: "Formatted at 2026-06-19 10:19:53 on dist-test-slave-6pt2"
server_key: "96107af8396e60298dc6ab4457b9e42a"
server_key_iv: "8ffac846010f0255650acda1a7ead878"
server_key_version: "encryptionkey@0"
I20260619 10:19:53.107193 16206 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781864224577058-825-0/minicluster-data/ts-0/wal/instance:
uuid: "d835bbc06b3d47f7b10e69f31a298ed9"
format_stamp: "Formatted at 2026-06-19 10:19:53 on dist-test-slave-6pt2"
server_key: "96107af8396e60298dc6ab4457b9e42a"
server_key_iv: "8ffac846010f0255650acda1a7ead878"
server_key_version: "encryptionkey@0"
I20260619 10:19:53.108418 16206 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 10:19:53.109095 16221 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 10:19:53.109264 16206 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 10:19:53.109360 16206 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781864224577058-825-0/minicluster-data/ts-0/data,/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781864224577058-825-0/minicluster-data/ts-0/wal
uuid: "d835bbc06b3d47f7b10e69f31a298ed9"
format_stamp: "Formatted at 2026-06-19 10:19:53 on dist-test-slave-6pt2"
server_key: "96107af8396e60298dc6ab4457b9e42a"
server_key_iv: "8ffac846010f0255650acda1a7ead878"
server_key_version: "encryptionkey@0"
I20260619 10:19:53.109436 16206 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781864224577058-825-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781864224577058-825-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781864224577058-825-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 10:19:53.133635 16206 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 10:19:53.136020 16206 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 10:19:53.136202 16206 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 10:19:53.136441 16206 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 10:19:53.136744 16206 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 10:19:53.136797 16206 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 10:19:53.136838 16206 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 10:19:53.136866 16206 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 10:19:53.142582 16206 rpc_server.cc:307] RPC server started. Bound to: 127.0.206.65:41351
I20260619 10:19:53.142645 16334 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.206.65:41351 every 8 connection(s)
I20260619 10:19:53.142943 16206 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781864224577058-825-0/minicluster-data/ts-0/data/info.pb
I20260619 10:19:53.147861 825 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTU9OP6/build/release/bin/kudu as pid 16206
I20260619 10:19:53.147982 825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781864224577058-825-0/minicluster-data/ts-0/wal/instance
I20260619 10:19:53.148193 825 external_mini_cluster.cc:1428] Setting key bc3a50d213444a03a7ec816e7d93ce00
Jun 19 10:19:53 dist-test-slave-6pt2 krb5kdc[16099](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864393, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.65@KRBTEST.COM for kudu/127.0.206.126@KRBTEST.COM
I20260619 10:19:53.153403 16191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:19:53.143606 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.65:48963 (local address 127.0.206.126:33511)
0619 10:19:53.143672 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0619 10:19:53.143675 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 10:19:53.143905 (+ 230us) server_negotiation.cc:407] Connection header received
0619 10:19:53.144360 (+ 455us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:19:53.144361 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:19:53.144374 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:19:53.144425 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:19:53.145310 (+ 885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:19:53.145812 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:19:53.147038 (+ 1226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:19:53.147256 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:19:53.147313 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:19:53.149444 (+ 2131us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 10:19:53.149446 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 10:19:53.149446 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 10:19:53.149454 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 10:19:53.151615 (+ 2161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:19:53.152366 (+ 751us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:19:53.152367 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:19:53.152367 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:19:53.152430 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:19:53.152714 (+ 284us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:19:53.152715 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:19:53.152716 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:19:53.152854 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 10:19:53.152929 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0619 10:19:53.153229 (+ 300us) server_negotiation.cc:299] Negotiation successful
0619 10:19:53.153319 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260619 10:19:53.153595 16337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:19:53.143592 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:33511 (local address 127.0.206.65:48963)
0619 10:19:53.143783 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0619 10:19:53.143819 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0619 10:19:53.144271 (+ 452us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 10:19:53.144474 (+ 203us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 10:19:53.144477 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 10:19:53.145048 (+ 571us) client_negotiation.cc:189] Negotiated authn=SASL
0619 10:19:53.145239 (+ 191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:19:53.145243 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:19:53.145900 (+ 657us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:19:53.145901 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:19:53.146922 (+ 1021us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:19:53.146925 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:19:53.147327 (+ 402us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:19:53.147328 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:19:53.147425 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:19:53.147935 (+ 510us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 10:19:53.147943 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 10:19:53.149350 (+ 1407us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 10:19:53.151809 (+ 2459us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 10:19:53.151816 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 10:19:53.151818 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 10:19:53.152265 (+ 447us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 10:19:53.152519 (+ 254us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 10:19:53.152520 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 10:19:53.152520 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 10:19:53.152653 (+ 133us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 10:19:53.152933 (+ 280us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 10:19:53.152939 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 10:19:53.153070 (+ 131us) client_negotiation.cc:769] Sending connection context
0619 10:19:53.153203 (+ 133us) client_negotiation.cc:240] Negotiation successful
0619 10:19:53.153348 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":109,"threads_started":1}
I20260619 10:19:53.154026 16335 heartbeater.cc:347] Connected to a master server at 127.0.206.126:33511
I20260619 10:19:53.154134 16335 heartbeater.cc:464] Registering TS with master...
I20260619 10:19:53.154304 16335 heartbeater.cc:511] Master 127.0.206.126:33511 requested a full tablet report, sending...
I20260619 10:19:53.154831 16144 ts_manager.cc:194] Registered new tserver with Master: d835bbc06b3d47f7b10e69f31a298ed9 (127.0.206.65:41351)
I20260619 10:19:53.155423 16144 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.206.65@KRBTEST.COM'} at 127.0.206.65:48963
WARNING: no policy specified for kudu/127.0.206.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.206.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.206.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781864224577058-825-0/krb5kdc/kudu_127.0.206.66.keytab.
Entry for principal kudu/127.0.206.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781864224577058-825-0/krb5kdc/kudu_127.0.206.66.keytab.
I20260619 10:19:53.171264 825 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTU9OP6/build/release/bin/kudu
/tmp/dist-test-taskTU9OP6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781864224577058-825-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781864224577058-825-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781864224577058-825-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781864224577058-825-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.206.66:0
--local_ip_for_outbound_sockets=127.0.206.66
--webserver_interface=127.0.206.66
--webserver_port=0
--tserver_master_addrs=127.0.206.126:33511
--keytab_file=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781864224577058-825-0/krb5kdc/kudu_127.0.206.66.keytab
--principal=kudu/127.0.206.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.206.84:43867
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781864224577058-825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781864224577058-825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781864224577058-825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 10:19:53.248282 16340 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 10:19:53.248463 16340 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 10:19:53.248486 16340 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 10:19:53.249560 16340 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 10:19:53.249594 16340 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 10:19:53.249619 16340 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.206.66
I20260619 10:19:53.250923 16340 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.206.84:43867
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781864224577058-825-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781864224577058-825-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781864224577058-825-0/krb5kdc/kudu_127.0.206.66.keytab
--principal=kudu/127.0.206.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.206.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781864224577058-825-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.206.66
--webserver_port=0
--tserver_master_addrs=127.0.206.126:33511
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16340
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.206.66
--log_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781864224577058-825-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 09:32:44 UTC on ccycloud.quasar-gbtldp.root.comops.site
build id 4455440
I20260619 10:19:53.251103 16340 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 10:19:53.251259 16340 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 10:19:53.255738 16346 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 10:19:53.255733 16348 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 10:19:53.255733 16345 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 10:19:53.255762 16340 server_base.cc:1034] running on GCE node
I20260619 10:19:53.256137 16340 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 10:19:53.256371 16340 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 10:19:53.257505 16340 hybrid_clock.cc:648] HybridClock initialized: now 1781864393257489 us; error 37 us; skew 500 ppm
Jun 19 10:19:53 dist-test-slave-6pt2 krb5kdc[16099](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864393, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 10:19:53.259887 16340 init.cc:377] Logged in from keytab as kudu/127.0.206.66@KRBTEST.COM (short username kudu)
I20260619 10:19:53.260310 16340 webserver.cc:460] Webserver started at http://127.0.206.66:39951/ using document root <none> and password file <none>
I20260619 10:19:53.260510 16340 fs_manager.cc:359] Metadata directory not provided
I20260619 10:19:53.260551 16340 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 10:19:53.260705 16340 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 10:19:53.261646 16340 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781864224577058-825-0/minicluster-data/ts-1/data/instance:
uuid: "69870ff4ad6a44ecb692fb40428dec06"
format_stamp: "Formatted at 2026-06-19 10:19:53 on dist-test-slave-6pt2"
server_key: "47ac82c46eba2f3daf1ebc6486ee0b63"
server_key_iv: "76cc672c9e34ed3da1f5b39f9c85b5fd"
server_key_version: "encryptionkey@0"
I20260619 10:19:53.261976 16340 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781864224577058-825-0/minicluster-data/ts-1/wal/instance:
uuid: "69870ff4ad6a44ecb692fb40428dec06"
format_stamp: "Formatted at 2026-06-19 10:19:53 on dist-test-slave-6pt2"
server_key: "47ac82c46eba2f3daf1ebc6486ee0b63"
server_key_iv: "76cc672c9e34ed3da1f5b39f9c85b5fd"
server_key_version: "encryptionkey@0"
I20260619 10:19:53.263278 16340 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.002s
I20260619 10:19:53.263929 16355 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 10:19:53.264087 16340 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 10:19:53.264180 16340 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781864224577058-825-0/minicluster-data/ts-1/data,/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781864224577058-825-0/minicluster-data/ts-1/wal
uuid: "69870ff4ad6a44ecb692fb40428dec06"
format_stamp: "Formatted at 2026-06-19 10:19:53 on dist-test-slave-6pt2"
server_key: "47ac82c46eba2f3daf1ebc6486ee0b63"
server_key_iv: "76cc672c9e34ed3da1f5b39f9c85b5fd"
server_key_version: "encryptionkey@0"
I20260619 10:19:53.264252 16340 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781864224577058-825-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781864224577058-825-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781864224577058-825-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 10:19:53.273075 16340 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 10:19:53.274780 16340 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 10:19:53.274924 16340 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 10:19:53.275125 16340 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 10:19:53.275419 16340 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 10:19:53.275471 16340 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 10:19:53.275504 16340 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 10:19:53.275529 16340 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 10:19:53.281245 16340 rpc_server.cc:307] RPC server started. Bound to: 127.0.206.66:34349
I20260619 10:19:53.281293 16468 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.206.66:34349 every 8 connection(s)
I20260619 10:19:53.281565 16340 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781864224577058-825-0/minicluster-data/ts-1/data/info.pb
I20260619 10:19:53.285636 825 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTU9OP6/build/release/bin/kudu as pid 16340
I20260619 10:19:53.285719 825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781864224577058-825-0/minicluster-data/ts-1/wal/instance
I20260619 10:19:53.285948 825 external_mini_cluster.cc:1428] Setting key 6d86a8ee449005178534964eacc42149
Jun 19 10:19:53 dist-test-slave-6pt2 krb5kdc[16099](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864393, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.66@KRBTEST.COM for kudu/127.0.206.126@KRBTEST.COM
I20260619 10:19:53.291025 16191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:19:53.282240 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.66:60825 (local address 127.0.206.126:33511)
0619 10:19:53.282309 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0619 10:19:53.282311 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 10:19:53.282511 (+ 200us) server_negotiation.cc:407] Connection header received
0619 10:19:53.283053 (+ 542us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:19:53.283054 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:19:53.283068 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:19:53.283115 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:19:53.283996 (+ 881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:19:53.284492 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:19:53.285480 (+ 988us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:19:53.285710 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:19:53.285774 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:19:53.288236 (+ 2462us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 10:19:53.288237 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 10:19:53.288238 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 10:19:53.288246 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 10:19:53.289608 (+ 1362us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:19:53.290044 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:19:53.290045 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:19:53.290045 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:19:53.290104 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:19:53.290359 (+ 255us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:19:53.290360 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:19:53.290360 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:19:53.290471 (+ 111us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 10:19:53.290546 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0619 10:19:53.290849 (+ 303us) server_negotiation.cc:299] Negotiation successful
0619 10:19:53.290937 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260619 10:19:53.291211 16471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:19:53.282215 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:33511 (local address 127.0.206.66:60825)
0619 10:19:53.282387 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0619 10:19:53.282418 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0619 10:19:53.282939 (+ 521us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 10:19:53.283192 (+ 253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 10:19:53.283195 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 10:19:53.283720 (+ 525us) client_negotiation.cc:189] Negotiated authn=SASL
0619 10:19:53.283888 (+ 168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:19:53.283892 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:19:53.284605 (+ 713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:19:53.284606 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:19:53.285388 (+ 782us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:19:53.285390 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:19:53.285829 (+ 439us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:19:53.285830 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:19:53.285983 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:19:53.286488 (+ 505us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 10:19:53.286497 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0619 10:19:53.288137 (+ 1640us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 10:19:53.289712 (+ 1575us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 10:19:53.289719 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 10:19:53.289721 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 10:19:53.289954 (+ 233us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 10:19:53.290179 (+ 225us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 10:19:53.290180 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 10:19:53.290180 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 10:19:53.290287 (+ 107us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 10:19:53.290538 (+ 251us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 10:19:53.290544 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 10:19:53.290712 (+ 168us) client_negotiation.cc:769] Sending connection context
0619 10:19:53.290826 (+ 114us) client_negotiation.cc:240] Negotiation successful
0619 10:19:53.290973 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":87,"threads_started":1}
I20260619 10:19:53.291671 16469 heartbeater.cc:347] Connected to a master server at 127.0.206.126:33511
I20260619 10:19:53.291778 16469 heartbeater.cc:464] Registering TS with master...
I20260619 10:19:53.291972 16469 heartbeater.cc:511] Master 127.0.206.126:33511 requested a full tablet report, sending...
I20260619 10:19:53.292347 16144 ts_manager.cc:194] Registered new tserver with Master: 69870ff4ad6a44ecb692fb40428dec06 (127.0.206.66:34349)
I20260619 10:19:53.292727 16144 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.206.66@KRBTEST.COM'} at 127.0.206.66:60825
WARNING: no policy specified for kudu/127.0.206.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.206.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.206.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781864224577058-825-0/krb5kdc/kudu_127.0.206.67.keytab.
Entry for principal kudu/127.0.206.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781864224577058-825-0/krb5kdc/kudu_127.0.206.67.keytab.
I20260619 10:19:53.309712 825 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTU9OP6/build/release/bin/kudu
/tmp/dist-test-taskTU9OP6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781864224577058-825-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781864224577058-825-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781864224577058-825-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781864224577058-825-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.206.67:0
--local_ip_for_outbound_sockets=127.0.206.67
--webserver_interface=127.0.206.67
--webserver_port=0
--tserver_master_addrs=127.0.206.126:33511
--keytab_file=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781864224577058-825-0/krb5kdc/kudu_127.0.206.67.keytab
--principal=kudu/127.0.206.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.206.84:43867
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781864224577058-825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781864224577058-825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781864224577058-825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 10:19:53.386885 16474 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 10:19:53.387061 16474 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 10:19:53.387084 16474 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 10:19:53.388213 16474 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 10:19:53.388252 16474 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 10:19:53.388278 16474 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.206.67
I20260619 10:19:53.389654 16474 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.206.84:43867
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781864224577058-825-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781864224577058-825-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781864224577058-825-0/krb5kdc/kudu_127.0.206.67.keytab
--principal=kudu/127.0.206.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.206.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781864224577058-825-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.206.67
--webserver_port=0
--tserver_master_addrs=127.0.206.126:33511
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16474
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.206.67
--log_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781864224577058-825-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 09:32:44 UTC on ccycloud.quasar-gbtldp.root.comops.site
build id 4455440
I20260619 10:19:53.389880 16474 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 10:19:53.390084 16474 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 10:19:53.394954 16480 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 10:19:53.394987 16479 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 10:19:53.394919 16482 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 10:19:53.395133 16474 server_base.cc:1034] running on GCE node
I20260619 10:19:53.395363 16474 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 10:19:53.395625 16474 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 10:19:53.396766 16474 hybrid_clock.cc:648] HybridClock initialized: now 1781864393396754 us; error 28 us; skew 500 ppm
Jun 19 10:19:53 dist-test-slave-6pt2 krb5kdc[16099](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864393, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 10:19:53.399232 16474 init.cc:377] Logged in from keytab as kudu/127.0.206.67@KRBTEST.COM (short username kudu)
I20260619 10:19:53.399677 16474 webserver.cc:460] Webserver started at http://127.0.206.67:33907/ using document root <none> and password file <none>
I20260619 10:19:53.399904 16474 fs_manager.cc:359] Metadata directory not provided
I20260619 10:19:53.399948 16474 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 10:19:53.400089 16474 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 10:19:53.401082 16474 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781864224577058-825-0/minicluster-data/ts-2/data/instance:
uuid: "c1b395dd05a64924a62063d847ac8bb2"
format_stamp: "Formatted at 2026-06-19 10:19:53 on dist-test-slave-6pt2"
server_key: "b86270f85a9aa898f56881c27592156e"
server_key_iv: "e0fdc184a4f5eb447f166ff615c876b4"
server_key_version: "encryptionkey@0"
I20260619 10:19:53.401444 16474 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781864224577058-825-0/minicluster-data/ts-2/wal/instance:
uuid: "c1b395dd05a64924a62063d847ac8bb2"
format_stamp: "Formatted at 2026-06-19 10:19:53 on dist-test-slave-6pt2"
server_key: "b86270f85a9aa898f56881c27592156e"
server_key_iv: "e0fdc184a4f5eb447f166ff615c876b4"
server_key_version: "encryptionkey@0"
I20260619 10:19:53.402750 16474 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 10:19:53.403452 16489 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 10:19:53.403627 16474 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 10:19:53.403714 16474 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781864224577058-825-0/minicluster-data/ts-2/data,/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781864224577058-825-0/minicluster-data/ts-2/wal
uuid: "c1b395dd05a64924a62063d847ac8bb2"
format_stamp: "Formatted at 2026-06-19 10:19:53 on dist-test-slave-6pt2"
server_key: "b86270f85a9aa898f56881c27592156e"
server_key_iv: "e0fdc184a4f5eb447f166ff615c876b4"
server_key_version: "encryptionkey@0"
I20260619 10:19:53.403793 16474 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781864224577058-825-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781864224577058-825-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781864224577058-825-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 10:19:53.430209 16474 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 10:19:53.432090 16474 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 10:19:53.432278 16474 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 10:19:53.432502 16474 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 10:19:53.432788 16474 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 10:19:53.432837 16474 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 10:19:53.432878 16474 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 10:19:53.432904 16474 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 10:19:53.438452 16474 rpc_server.cc:307] RPC server started. Bound to: 127.0.206.67:38085
I20260619 10:19:53.438495 16602 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.206.67:38085 every 8 connection(s)
I20260619 10:19:53.438838 16474 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781864224577058-825-0/minicluster-data/ts-2/data/info.pb
I20260619 10:19:53.444167 825 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTU9OP6/build/release/bin/kudu as pid 16474
I20260619 10:19:53.444288 825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781864224577058-825-0/minicluster-data/ts-2/wal/instance
I20260619 10:19:53.444482 825 external_mini_cluster.cc:1428] Setting key 92485ad270b082b2df42abe85fb83f44
Jun 19 10:19:53 dist-test-slave-6pt2 krb5kdc[16099](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864393, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.67@KRBTEST.COM for kudu/127.0.206.126@KRBTEST.COM
I20260619 10:19:53.448302 16191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:19:53.439528 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.67:49715 (local address 127.0.206.126:33511)
0619 10:19:53.439582 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0619 10:19:53.439584 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 10:19:53.439787 (+ 203us) server_negotiation.cc:407] Connection header received
0619 10:19:53.440239 (+ 452us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:19:53.440240 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:19:53.440249 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:19:53.440309 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:19:53.441144 (+ 835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:19:53.441616 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:19:53.442533 (+ 917us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:19:53.442801 (+ 268us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:19:53.442870 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:19:53.445295 (+ 2425us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 10:19:53.445296 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 10:19:53.445297 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 10:19:53.445305 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 10:19:53.446937 (+ 1632us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:19:53.447463 (+ 526us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:19:53.447464 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:19:53.447465 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:19:53.447516 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:19:53.447744 (+ 228us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:19:53.447745 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:19:53.447746 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:19:53.447914 (+ 168us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 10:19:53.447982 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0619 10:19:53.448178 (+ 196us) server_negotiation.cc:299] Negotiation successful
0619 10:19:53.448235 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260619 10:19:53.448523 16605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:19:53.439504 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:33511 (local address 127.0.206.67:49715)
0619 10:19:53.439669 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0619 10:19:53.439708 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0619 10:19:53.440161 (+ 453us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 10:19:53.440369 (+ 208us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 10:19:53.440372 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 10:19:53.440900 (+ 528us) client_negotiation.cc:189] Negotiated authn=SASL
0619 10:19:53.441074 (+ 174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:19:53.441079 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:19:53.441693 (+ 614us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:19:53.441694 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:19:53.442464 (+ 770us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:19:53.442466 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:19:53.442882 (+ 416us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:19:53.442883 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:19:53.443025 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:19:53.443552 (+ 527us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 10:19:53.443562 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0619 10:19:53.445195 (+ 1633us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 10:19:53.447038 (+ 1843us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 10:19:53.447044 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 10:19:53.447046 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 10:19:53.447401 (+ 355us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 10:19:53.447574 (+ 173us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 10:19:53.447574 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 10:19:53.447574 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 10:19:53.447664 (+ 90us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 10:19:53.447979 (+ 315us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 10:19:53.447983 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 10:19:53.448086 (+ 103us) client_negotiation.cc:769] Sending connection context
0619 10:19:53.448171 (+ 85us) client_negotiation.cc:240] Negotiation successful
0619 10:19:53.448282 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":94,"threads_started":1}
I20260619 10:19:53.448949 16603 heartbeater.cc:347] Connected to a master server at 127.0.206.126:33511
I20260619 10:19:53.449024 16603 heartbeater.cc:464] Registering TS with master...
I20260619 10:19:53.449221 16603 heartbeater.cc:511] Master 127.0.206.126:33511 requested a full tablet report, sending...
I20260619 10:19:53.449582 16144 ts_manager.cc:194] Registered new tserver with Master: c1b395dd05a64924a62063d847ac8bb2 (127.0.206.67:38085)
I20260619 10:19:53.449942 16144 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.206.67@KRBTEST.COM'} at 127.0.206.67:49715
I20260619 10:19:53.456089 825 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 10:19:53.462960 16191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:19:53.457406 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33318 (local address 127.0.206.126:33511)
0619 10:19:53.457475 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0619 10:19:53.457477 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 10:19:53.457575 (+ 98us) server_negotiation.cc:407] Connection header received
0619 10:19:53.457612 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:19:53.457613 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:19:53.457621 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:19:53.457665 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:19:53.458242 (+ 577us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:19:53.458795 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:19:53.459453 (+ 658us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:19:53.459618 (+ 165us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:19:53.459663 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:19:53.460717 (+ 1054us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 10:19:53.460718 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 10:19:53.460718 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 10:19:53.460723 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0619 10:19:53.462049 (+ 1326us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:19:53.462320 (+ 271us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:19:53.462321 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:19:53.462321 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:19:53.462367 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:19:53.462476 (+ 109us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:19:53.462477 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:19:53.462477 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:19:53.462591 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 10:19:53.462709 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0619 10:19:53.462798 (+ 89us) server_negotiation.cc:299] Negotiation successful
0619 10:19:53.462854 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260619 10:19:53.464470 16144 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33318:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 10:19:53.465893 16144 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 10:19:53.472319 16611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:19:53.467889 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.66:34349 (local address 127.0.0.1:44370)
0619 10:19:53.468120 (+ 231us) negotiation.cc:107] Waiting for socket to connect
0619 10:19:53.468129 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0619 10:19:53.468169 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 10:19:53.468451 (+ 282us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 10:19:53.468453 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 10:19:53.468458 (+ 5us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 10:19:53.468520 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:19:53.468522 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:19:53.469796 (+ 1274us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:19:53.469796 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:19:53.471096 (+ 1300us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:19:53.471097 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:19:53.472011 (+ 914us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:19:53.472012 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:19:53.472088 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:19:53.472101 (+ 13us) client_negotiation.cc:769] Sending connection context
0619 10:19:53.472183 (+ 82us) client_negotiation.cc:240] Negotiation successful
0619 10:19:53.472247 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":220,"mutex_wait_us":10,"thread_start_us":59,"threads_started":1}
I20260619 10:19:53.472352 16616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:19:53.468099 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44370 (local address 127.0.206.66:34349)
0619 10:19:53.468255 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0619 10:19:53.468260 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 10:19:53.468284 (+ 24us) server_negotiation.cc:407] Connection header received
0619 10:19:53.468327 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:19:53.468329 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:19:53.468346 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:19:53.468437 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 10:19:53.468597 (+ 160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:19:53.469701 (+ 1104us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:19:53.471186 (+ 1485us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:19:53.471920 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:19:53.471993 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:19:53.472033 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0619 10:19:53.472219 (+ 186us) server_negotiation.cc:299] Negotiation successful
0619 10:19:53.472276 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":123,"thread_start_us":68,"threads_started":1}
I20260619 10:19:53.472954 16613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:19:53.467865 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.65:41351 (local address 127.0.0.1:36104)
0619 10:19:53.468020 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0619 10:19:53.468059 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0619 10:19:53.468122 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 10:19:53.468451 (+ 329us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 10:19:53.468453 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 10:19:53.468455 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 10:19:53.468520 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:19:53.468521 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:19:53.470784 (+ 2263us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:19:53.470785 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:19:53.471779 (+ 994us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:19:53.471781 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:19:53.472686 (+ 905us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:19:53.472688 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:19:53.472769 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:19:53.472773 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 10:19:53.472828 (+ 55us) client_negotiation.cc:240] Negotiation successful
0619 10:19:53.472871 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":132,"mutex_wait_us":5,"thread_start_us":54,"threads_started":1}
I20260619 10:19:53.472968 16614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:19:53.467970 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36104 (local address 127.0.206.65:41351)
0619 10:19:53.468283 (+ 313us) server_negotiation.cc:206] Beginning negotiation
0619 10:19:53.468287 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 10:19:53.468297 (+ 10us) server_negotiation.cc:407] Connection header received
0619 10:19:53.468331 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:19:53.468333 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:19:53.468347 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:19:53.468437 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 10:19:53.469830 (+ 1393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:19:53.470667 (+ 837us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:19:53.471864 (+ 1197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:19:53.472598 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:19:53.472684 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:19:53.472739 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0619 10:19:53.472851 (+ 112us) server_negotiation.cc:299] Negotiation successful
0619 10:19:53.472895 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":276,"thread_start_us":74,"threads_started":1}
I20260619 10:19:53.473371 16612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:19:53.467892 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.67:38085 (local address 127.0.0.1:44126)
0619 10:19:53.468232 (+ 340us) negotiation.cc:107] Waiting for socket to connect
0619 10:19:53.468239 (+ 7us) client_negotiation.cc:174] Beginning negotiation
0619 10:19:53.468272 (+ 33us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 10:19:53.468808 (+ 536us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 10:19:53.468809 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 10:19:53.468812 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 10:19:53.468857 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:19:53.468858 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:19:53.469991 (+ 1133us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:19:53.469991 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:19:53.471095 (+ 1104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:19:53.471097 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:19:53.473144 (+ 2047us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:19:53.473145 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:19:53.473203 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:19:53.473206 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 10:19:53.473266 (+ 60us) client_negotiation.cc:240] Negotiation successful
0619 10:19:53.473305 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":325,"thread_start_us":55,"threads_started":1}
I20260619 10:19:53.473407 16615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:19:53.468074 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44126 (local address 127.0.206.67:38085)
0619 10:19:53.468621 (+ 547us) server_negotiation.cc:206] Beginning negotiation
0619 10:19:53.468626 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 10:19:53.468637 (+ 11us) server_negotiation.cc:407] Connection header received
0619 10:19:53.468673 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:19:53.468675 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:19:53.468690 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:19:53.468775 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 10:19:53.468908 (+ 133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:19:53.469896 (+ 988us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:19:53.472061 (+ 2165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:19:53.472765 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:19:53.473140 (+ 375us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:19:53.473202 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0619 10:19:53.473284 (+ 82us) server_negotiation.cc:299] Negotiation successful
0619 10:19:53.473331 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":457,"thread_start_us":79,"threads_started":1}
I20260619 10:19:53.473315 16403 tablet_service.cc:1467] Processing CreateTablet for tablet 20be40cde00c432cbd3abb7e3fd762a8 (DEFAULT_TABLE table=test-table [id=8381967a37ca41fd9f78949ad371851e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 10:19:53.473656 16403 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 20be40cde00c432cbd3abb7e3fd762a8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 10:19:53.473799 16269 tablet_service.cc:1467] Processing CreateTablet for tablet 20be40cde00c432cbd3abb7e3fd762a8 (DEFAULT_TABLE table=test-table [id=8381967a37ca41fd9f78949ad371851e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 10:19:53.474143 16269 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 20be40cde00c432cbd3abb7e3fd762a8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 10:19:53.474346 16537 tablet_service.cc:1467] Processing CreateTablet for tablet 20be40cde00c432cbd3abb7e3fd762a8 (DEFAULT_TABLE table=test-table [id=8381967a37ca41fd9f78949ad371851e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 10:19:53.474655 16537 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 20be40cde00c432cbd3abb7e3fd762a8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 10:19:53.476346 16617 tablet_bootstrap.cc:492] T 20be40cde00c432cbd3abb7e3fd762a8 P 69870ff4ad6a44ecb692fb40428dec06: Bootstrap starting.
I20260619 10:19:53.476899 16617 tablet_bootstrap.cc:654] T 20be40cde00c432cbd3abb7e3fd762a8 P 69870ff4ad6a44ecb692fb40428dec06: Neither blocks nor log segments found. Creating new log.
I20260619 10:19:53.477351 16617 log.cc:824] T 20be40cde00c432cbd3abb7e3fd762a8 P 69870ff4ad6a44ecb692fb40428dec06: Log is configured to *not* fsync() on all Append() calls
I20260619 10:19:53.477425 16618 tablet_bootstrap.cc:492] T 20be40cde00c432cbd3abb7e3fd762a8 P d835bbc06b3d47f7b10e69f31a298ed9: Bootstrap starting.
I20260619 10:19:53.477952 16620 tablet_bootstrap.cc:492] T 20be40cde00c432cbd3abb7e3fd762a8 P c1b395dd05a64924a62063d847ac8bb2: Bootstrap starting.
I20260619 10:19:53.478008 16618 tablet_bootstrap.cc:654] T 20be40cde00c432cbd3abb7e3fd762a8 P d835bbc06b3d47f7b10e69f31a298ed9: Neither blocks nor log segments found. Creating new log.
I20260619 10:19:53.478628 16618 log.cc:824] T 20be40cde00c432cbd3abb7e3fd762a8 P d835bbc06b3d47f7b10e69f31a298ed9: Log is configured to *not* fsync() on all Append() calls
I20260619 10:19:53.478685 16620 tablet_bootstrap.cc:654] T 20be40cde00c432cbd3abb7e3fd762a8 P c1b395dd05a64924a62063d847ac8bb2: Neither blocks nor log segments found. Creating new log.
I20260619 10:19:53.479259 16620 log.cc:824] T 20be40cde00c432cbd3abb7e3fd762a8 P c1b395dd05a64924a62063d847ac8bb2: Log is configured to *not* fsync() on all Append() calls
I20260619 10:19:53.479310 16617 tablet_bootstrap.cc:492] T 20be40cde00c432cbd3abb7e3fd762a8 P 69870ff4ad6a44ecb692fb40428dec06: No bootstrap required, opened a new log
I20260619 10:19:53.479346 16618 tablet_bootstrap.cc:492] T 20be40cde00c432cbd3abb7e3fd762a8 P d835bbc06b3d47f7b10e69f31a298ed9: No bootstrap required, opened a new log
I20260619 10:19:53.479364 16617 ts_tablet_manager.cc:1397] T 20be40cde00c432cbd3abb7e3fd762a8 P 69870ff4ad6a44ecb692fb40428dec06: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260619 10:19:53.479451 16618 ts_tablet_manager.cc:1397] T 20be40cde00c432cbd3abb7e3fd762a8 P d835bbc06b3d47f7b10e69f31a298ed9: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260619 10:19:53.479995 16620 tablet_bootstrap.cc:492] T 20be40cde00c432cbd3abb7e3fd762a8 P c1b395dd05a64924a62063d847ac8bb2: No bootstrap required, opened a new log
I20260619 10:19:53.480067 16620 ts_tablet_manager.cc:1397] T 20be40cde00c432cbd3abb7e3fd762a8 P c1b395dd05a64924a62063d847ac8bb2: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260619 10:19:53.480916 16618 raft_consensus.cc:348] T 20be40cde00c432cbd3abb7e3fd762a8 P d835bbc06b3d47f7b10e69f31a298ed9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d835bbc06b3d47f7b10e69f31a298ed9" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 41351 } } peers { permanent_uuid: "69870ff4ad6a44ecb692fb40428dec06" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 34349 } } peers { permanent_uuid: "c1b395dd05a64924a62063d847ac8bb2" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 38085 } }
I20260619 10:19:53.481074 16618 raft_consensus.cc:374] T 20be40cde00c432cbd3abb7e3fd762a8 P d835bbc06b3d47f7b10e69f31a298ed9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 10:19:53.481134 16618 raft_consensus.cc:729] T 20be40cde00c432cbd3abb7e3fd762a8 P d835bbc06b3d47f7b10e69f31a298ed9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d835bbc06b3d47f7b10e69f31a298ed9, State: Initialized, Role: FOLLOWER
I20260619 10:19:53.481304 16618 consensus_queue.cc:260] T 20be40cde00c432cbd3abb7e3fd762a8 P d835bbc06b3d47f7b10e69f31a298ed9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d835bbc06b3d47f7b10e69f31a298ed9" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 41351 } } peers { permanent_uuid: "69870ff4ad6a44ecb692fb40428dec06" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 34349 } } peers { permanent_uuid: "c1b395dd05a64924a62063d847ac8bb2" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 38085 } }
I20260619 10:19:53.481269 16617 raft_consensus.cc:348] T 20be40cde00c432cbd3abb7e3fd762a8 P 69870ff4ad6a44ecb692fb40428dec06 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d835bbc06b3d47f7b10e69f31a298ed9" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 41351 } } peers { permanent_uuid: "69870ff4ad6a44ecb692fb40428dec06" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 34349 } } peers { permanent_uuid: "c1b395dd05a64924a62063d847ac8bb2" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 38085 } }
I20260619 10:19:53.481464 16617 raft_consensus.cc:374] T 20be40cde00c432cbd3abb7e3fd762a8 P 69870ff4ad6a44ecb692fb40428dec06 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 10:19:53.481518 16617 raft_consensus.cc:729] T 20be40cde00c432cbd3abb7e3fd762a8 P 69870ff4ad6a44ecb692fb40428dec06 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 69870ff4ad6a44ecb692fb40428dec06, State: Initialized, Role: FOLLOWER
I20260619 10:19:53.481722 16335 heartbeater.cc:503] Master 127.0.206.126:33511 was elected leader, sending a full tablet report...
I20260619 10:19:53.481729 16618 ts_tablet_manager.cc:1428] T 20be40cde00c432cbd3abb7e3fd762a8 P d835bbc06b3d47f7b10e69f31a298ed9: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260619 10:19:53.481649 16617 consensus_queue.cc:260] T 20be40cde00c432cbd3abb7e3fd762a8 P 69870ff4ad6a44ecb692fb40428dec06 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d835bbc06b3d47f7b10e69f31a298ed9" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 41351 } } peers { permanent_uuid: "69870ff4ad6a44ecb692fb40428dec06" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 34349 } } peers { permanent_uuid: "c1b395dd05a64924a62063d847ac8bb2" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 38085 } }
I20260619 10:19:53.482041 16620 raft_consensus.cc:348] T 20be40cde00c432cbd3abb7e3fd762a8 P c1b395dd05a64924a62063d847ac8bb2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d835bbc06b3d47f7b10e69f31a298ed9" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 41351 } } peers { permanent_uuid: "69870ff4ad6a44ecb692fb40428dec06" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 34349 } } peers { permanent_uuid: "c1b395dd05a64924a62063d847ac8bb2" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 38085 } }
I20260619 10:19:53.482250 16620 raft_consensus.cc:374] T 20be40cde00c432cbd3abb7e3fd762a8 P c1b395dd05a64924a62063d847ac8bb2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 10:19:53.482283 16617 ts_tablet_manager.cc:1428] T 20be40cde00c432cbd3abb7e3fd762a8 P 69870ff4ad6a44ecb692fb40428dec06: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260619 10:19:53.482306 16620 raft_consensus.cc:729] T 20be40cde00c432cbd3abb7e3fd762a8 P c1b395dd05a64924a62063d847ac8bb2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c1b395dd05a64924a62063d847ac8bb2, State: Initialized, Role: FOLLOWER
I20260619 10:19:53.482389 16469 heartbeater.cc:503] Master 127.0.206.126:33511 was elected leader, sending a full tablet report...
I20260619 10:19:53.482434 16620 consensus_queue.cc:260] T 20be40cde00c432cbd3abb7e3fd762a8 P c1b395dd05a64924a62063d847ac8bb2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d835bbc06b3d47f7b10e69f31a298ed9" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 41351 } } peers { permanent_uuid: "69870ff4ad6a44ecb692fb40428dec06" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 34349 } } peers { permanent_uuid: "c1b395dd05a64924a62063d847ac8bb2" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 38085 } }
I20260619 10:19:53.482743 16620 ts_tablet_manager.cc:1428] T 20be40cde00c432cbd3abb7e3fd762a8 P c1b395dd05a64924a62063d847ac8bb2: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260619 10:19:53.482771 16603 heartbeater.cc:503] Master 127.0.206.126:33511 was elected leader, sending a full tablet report...
W20260619 10:19:53.532176 16470 tablet.cc:2369] T 20be40cde00c432cbd3abb7e3fd762a8 P 69870ff4ad6a44ecb692fb40428dec06: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 10:19:53.572273 16624 raft_consensus.cc:482] T 20be40cde00c432cbd3abb7e3fd762a8 P 69870ff4ad6a44ecb692fb40428dec06 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 10:19:53.572438 16624 raft_consensus.cc:504] T 20be40cde00c432cbd3abb7e3fd762a8 P 69870ff4ad6a44ecb692fb40428dec06 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d835bbc06b3d47f7b10e69f31a298ed9" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 41351 } } peers { permanent_uuid: "69870ff4ad6a44ecb692fb40428dec06" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 34349 } } peers { permanent_uuid: "c1b395dd05a64924a62063d847ac8bb2" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 38085 } }
I20260619 10:19:53.572826 16624 leader_election.cc:288] T 20be40cde00c432cbd3abb7e3fd762a8 P 69870ff4ad6a44ecb692fb40428dec06 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d835bbc06b3d47f7b10e69f31a298ed9 (127.0.206.65:41351), c1b395dd05a64924a62063d847ac8bb2 (127.0.206.67:38085)
I20260619 10:19:53.575702 16614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:19:53.573190 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.66:49427 (local address 127.0.206.65:41351)
0619 10:19:53.573239 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0619 10:19:53.573241 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 10:19:53.573248 (+ 7us) server_negotiation.cc:407] Connection header received
0619 10:19:53.573278 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:19:53.573279 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:19:53.573286 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:19:53.573353 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 10:19:53.573508 (+ 155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:19:53.573985 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:19:53.574973 (+ 988us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:19:53.575363 (+ 390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:19:53.575446 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:19:53.575455 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0619 10:19:53.575606 (+ 151us) server_negotiation.cc:299] Negotiation successful
0619 10:19:53.575645 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260619 10:19:53.575695 16471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:19:53.573008 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.65:41351 (local address 127.0.206.66:49427)
0619 10:19:53.573075 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0619 10:19:53.573087 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 10:19:53.573145 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 10:19:53.573357 (+ 212us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 10:19:53.573358 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 10:19:53.573360 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 10:19:53.573437 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:19:53.573438 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:19:53.574069 (+ 631us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:19:53.574070 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:19:53.574885 (+ 815us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:19:53.574887 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:19:53.575452 (+ 565us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:19:53.575453 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:19:53.575527 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:19:53.575530 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 10:19:53.575589 (+ 59us) client_negotiation.cc:240] Negotiation successful
0619 10:19:53.575627 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":45}
I20260619 10:19:53.575829 16626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:19:53.573008 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.67:38085 (local address 127.0.206.66:33675)
0619 10:19:53.573325 (+ 317us) negotiation.cc:107] Waiting for socket to connect
0619 10:19:53.573334 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0619 10:19:53.573415 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 10:19:53.573557 (+ 142us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 10:19:53.573558 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 10:19:53.573559 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 10:19:53.573629 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:19:53.573630 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:19:53.574268 (+ 638us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:19:53.574269 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:19:53.574999 (+ 730us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:19:53.575001 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:19:53.575558 (+ 557us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:19:53.575559 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:19:53.575629 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:19:53.575633 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 10:19:53.575714 (+ 81us) client_negotiation.cc:240] Negotiation successful
0619 10:19:53.575749 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":300,"mutex_wait_us":106,"thread_start_us":155,"threads_started":1}
I20260619 10:19:53.575821 16615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:19:53.573119 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.66:33675 (local address 127.0.206.67:38085)
0619 10:19:53.573194 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0619 10:19:53.573197 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 10:19:53.573404 (+ 207us) server_negotiation.cc:407] Connection header received
0619 10:19:53.573481 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:19:53.573482 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:19:53.573490 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:19:53.573554 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 10:19:53.573722 (+ 168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:19:53.574175 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:19:53.575096 (+ 921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:19:53.575478 (+ 382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:19:53.575558 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:19:53.575566 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0619 10:19:53.575725 (+ 159us) server_negotiation.cc:299] Negotiation successful
0619 10:19:53.575764 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260619 10:19:53.576035 16289 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "20be40cde00c432cbd3abb7e3fd762a8" candidate_uuid: "69870ff4ad6a44ecb692fb40428dec06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d835bbc06b3d47f7b10e69f31a298ed9" is_pre_election: true
I20260619 10:19:53.576141 16557 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "20be40cde00c432cbd3abb7e3fd762a8" candidate_uuid: "69870ff4ad6a44ecb692fb40428dec06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c1b395dd05a64924a62063d847ac8bb2" is_pre_election: true
I20260619 10:19:53.576185 16289 raft_consensus.cc:2454] T 20be40cde00c432cbd3abb7e3fd762a8 P d835bbc06b3d47f7b10e69f31a298ed9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 69870ff4ad6a44ecb692fb40428dec06 in term 0.
I20260619 10:19:53.576295 16557 raft_consensus.cc:2454] T 20be40cde00c432cbd3abb7e3fd762a8 P c1b395dd05a64924a62063d847ac8bb2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 69870ff4ad6a44ecb692fb40428dec06 in term 0.
I20260619 10:19:53.576431 16356 leader_election.cc:302] T 20be40cde00c432cbd3abb7e3fd762a8 P 69870ff4ad6a44ecb692fb40428dec06 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 69870ff4ad6a44ecb692fb40428dec06, d835bbc06b3d47f7b10e69f31a298ed9; no voters:
I20260619 10:19:53.576570 16624 raft_consensus.cc:2781] T 20be40cde00c432cbd3abb7e3fd762a8 P 69870ff4ad6a44ecb692fb40428dec06 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 10:19:53.576622 16624 raft_consensus.cc:482] T 20be40cde00c432cbd3abb7e3fd762a8 P 69870ff4ad6a44ecb692fb40428dec06 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 10:19:53.576676 16624 raft_consensus.cc:3037] T 20be40cde00c432cbd3abb7e3fd762a8 P 69870ff4ad6a44ecb692fb40428dec06 [term 0 FOLLOWER]: Advancing to term 1
I20260619 10:19:53.577390 16624 raft_consensus.cc:504] T 20be40cde00c432cbd3abb7e3fd762a8 P 69870ff4ad6a44ecb692fb40428dec06 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d835bbc06b3d47f7b10e69f31a298ed9" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 41351 } } peers { permanent_uuid: "69870ff4ad6a44ecb692fb40428dec06" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 34349 } } peers { permanent_uuid: "c1b395dd05a64924a62063d847ac8bb2" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 38085 } }
I20260619 10:19:53.577524 16624 leader_election.cc:288] T 20be40cde00c432cbd3abb7e3fd762a8 P 69870ff4ad6a44ecb692fb40428dec06 [CANDIDATE]: Term 1 election: Requested vote from peers d835bbc06b3d47f7b10e69f31a298ed9 (127.0.206.65:41351), c1b395dd05a64924a62063d847ac8bb2 (127.0.206.67:38085)
I20260619 10:19:53.577654 16289 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "20be40cde00c432cbd3abb7e3fd762a8" candidate_uuid: "69870ff4ad6a44ecb692fb40428dec06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d835bbc06b3d47f7b10e69f31a298ed9"
I20260619 10:19:53.577685 16557 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "20be40cde00c432cbd3abb7e3fd762a8" candidate_uuid: "69870ff4ad6a44ecb692fb40428dec06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c1b395dd05a64924a62063d847ac8bb2"
I20260619 10:19:53.577732 16289 raft_consensus.cc:3037] T 20be40cde00c432cbd3abb7e3fd762a8 P d835bbc06b3d47f7b10e69f31a298ed9 [term 0 FOLLOWER]: Advancing to term 1
I20260619 10:19:53.577749 16557 raft_consensus.cc:3037] T 20be40cde00c432cbd3abb7e3fd762a8 P c1b395dd05a64924a62063d847ac8bb2 [term 0 FOLLOWER]: Advancing to term 1
I20260619 10:19:53.578589 16289 raft_consensus.cc:2454] T 20be40cde00c432cbd3abb7e3fd762a8 P d835bbc06b3d47f7b10e69f31a298ed9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 69870ff4ad6a44ecb692fb40428dec06 in term 1.
I20260619 10:19:53.578589 16557 raft_consensus.cc:2454] T 20be40cde00c432cbd3abb7e3fd762a8 P c1b395dd05a64924a62063d847ac8bb2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 69870ff4ad6a44ecb692fb40428dec06 in term 1.
I20260619 10:19:53.578780 16356 leader_election.cc:302] T 20be40cde00c432cbd3abb7e3fd762a8 P 69870ff4ad6a44ecb692fb40428dec06 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 69870ff4ad6a44ecb692fb40428dec06, d835bbc06b3d47f7b10e69f31a298ed9; no voters:
I20260619 10:19:53.578873 16624 raft_consensus.cc:2781] T 20be40cde00c432cbd3abb7e3fd762a8 P 69870ff4ad6a44ecb692fb40428dec06 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 10:19:53.579007 16624 raft_consensus.cc:686] T 20be40cde00c432cbd3abb7e3fd762a8 P 69870ff4ad6a44ecb692fb40428dec06 [term 1 LEADER]: Becoming Leader. State: Replica: 69870ff4ad6a44ecb692fb40428dec06, State: Running, Role: LEADER
I20260619 10:19:53.579107 16624 consensus_queue.cc:237] T 20be40cde00c432cbd3abb7e3fd762a8 P 69870ff4ad6a44ecb692fb40428dec06 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d835bbc06b3d47f7b10e69f31a298ed9" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 41351 } } peers { permanent_uuid: "69870ff4ad6a44ecb692fb40428dec06" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 34349 } } peers { permanent_uuid: "c1b395dd05a64924a62063d847ac8bb2" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 38085 } }
I20260619 10:19:53.579893 16143 catalog_manager.cc:5495] T 20be40cde00c432cbd3abb7e3fd762a8 P 69870ff4ad6a44ecb692fb40428dec06 reported cstate change: term changed from 0 to 1, leader changed from <none> to 69870ff4ad6a44ecb692fb40428dec06 (127.0.206.66). New cstate: current_term: 1 leader_uuid: "69870ff4ad6a44ecb692fb40428dec06" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d835bbc06b3d47f7b10e69f31a298ed9" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 41351 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "69870ff4ad6a44ecb692fb40428dec06" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 34349 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c1b395dd05a64924a62063d847ac8bb2" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 38085 } health_report { overall_health: UNKNOWN } } }
I20260619 10:19:53.587826 16616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:19:53.585841 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44380 (local address 127.0.206.66:34349)
0619 10:19:53.585876 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0619 10:19:53.585878 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 10:19:53.585886 (+ 8us) server_negotiation.cc:407] Connection header received
0619 10:19:53.585942 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:19:53.585943 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:19:53.585955 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:19:53.586013 (+ 58us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 10:19:53.586107 (+ 94us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:19:53.586522 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:19:53.587239 (+ 717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:19:53.587403 (+ 164us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:19:53.587452 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:19:53.587571 (+ 119us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 10:19:53.587652 (+ 81us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 10:19:53.587707 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0619 10:19:53.587744 (+ 37us) server_negotiation.cc:299] Negotiation successful
0619 10:19:53.587759 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260619 10:19:53.589360 16557 raft_consensus.cc:1261] T 20be40cde00c432cbd3abb7e3fd762a8 P c1b395dd05a64924a62063d847ac8bb2 [term 1 FOLLOWER]: Refusing update from remote peer 69870ff4ad6a44ecb692fb40428dec06: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 10:19:53.589356 16289 raft_consensus.cc:1261] T 20be40cde00c432cbd3abb7e3fd762a8 P d835bbc06b3d47f7b10e69f31a298ed9 [term 1 FOLLOWER]: Refusing update from remote peer 69870ff4ad6a44ecb692fb40428dec06: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 10:19:53.589632 16627 consensus_queue.cc:1035] T 20be40cde00c432cbd3abb7e3fd762a8 P 69870ff4ad6a44ecb692fb40428dec06 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d835bbc06b3d47f7b10e69f31a298ed9" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 41351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 10:19:53.589766 16624 consensus_queue.cc:1035] T 20be40cde00c432cbd3abb7e3fd762a8 P 69870ff4ad6a44ecb692fb40428dec06 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1b395dd05a64924a62063d847ac8bb2" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 38085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 10:19:53.591837 16630 mvcc.cc:205] Tried to move back new op lower bound from 7298516556139704320 to 7298516556100599808. Current Snapshot: MvccSnapshot[applied={T|T < 7298516556139704320}]
I20260619 10:19:53.591904 16632 mvcc.cc:205] Tried to move back new op lower bound from 7298516556139704320 to 7298516556100599808. Current Snapshot: MvccSnapshot[applied={T|T < 7298516556139704320}]
I20260619 10:19:53.592648 16631 mvcc.cc:205] Tried to move back new op lower bound from 7298516556139704320 to 7298516556100599808. Current Snapshot: MvccSnapshot[applied={T|T < 7298516556139704320}]
I20260619 10:19:53.594259 16143 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33318:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 10:19:53.594340 16143 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33318:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 10:19:53.595053 16143 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b4330665f37e4f6693f0900bed7dcfe6: Sending DeleteTablet for 3 replicas of tablet 20be40cde00c432cbd3abb7e3fd762a8
I20260619 10:19:53.595388 16269 tablet_service.cc:1514] Processing DeleteTablet for tablet 20be40cde00c432cbd3abb7e3fd762a8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 10:19:53 UTC) from {username='kudu', principal='kudu/127.0.206.126@KRBTEST.COM'} at 127.0.0.1:36104
I20260619 10:19:53.595432 16537 tablet_service.cc:1514] Processing DeleteTablet for tablet 20be40cde00c432cbd3abb7e3fd762a8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 10:19:53 UTC) from {username='kudu', principal='kudu/127.0.206.126@KRBTEST.COM'} at 127.0.0.1:44126
I20260619 10:19:53.595440 16403 tablet_service.cc:1514] Processing DeleteTablet for tablet 20be40cde00c432cbd3abb7e3fd762a8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 10:19:53 UTC) from {username='kudu', principal='kudu/127.0.206.126@KRBTEST.COM'} at 127.0.0.1:44370
I20260619 10:19:53.595597 16640 tablet_replica.cc:330] T 20be40cde00c432cbd3abb7e3fd762a8 P d835bbc06b3d47f7b10e69f31a298ed9: stopping tablet replica
I20260619 10:19:53.595623 16641 tablet_replica.cc:330] T 20be40cde00c432cbd3abb7e3fd762a8 P c1b395dd05a64924a62063d847ac8bb2: stopping tablet replica
I20260619 10:19:53.595664 825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTU9OP6/build/release/bin/kudu with pid 16206
I20260619 10:19:53.595715 16641 raft_consensus.cc:2229] T 20be40cde00c432cbd3abb7e3fd762a8 P c1b395dd05a64924a62063d847ac8bb2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 10:19:53.595793 16641 raft_consensus.cc:2258] T 20be40cde00c432cbd3abb7e3fd762a8 P c1b395dd05a64924a62063d847ac8bb2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 10:19:53.596298 16642 tablet_replica.cc:330] T 20be40cde00c432cbd3abb7e3fd762a8 P 69870ff4ad6a44ecb692fb40428dec06: stopping tablet replica
I20260619 10:19:53.596439 16641 ts_tablet_manager.cc:1905] T 20be40cde00c432cbd3abb7e3fd762a8 P c1b395dd05a64924a62063d847ac8bb2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 10:19:53.596447 16642 raft_consensus.cc:2229] T 20be40cde00c432cbd3abb7e3fd762a8 P 69870ff4ad6a44ecb692fb40428dec06 [term 1 LEADER]: Raft consensus shutting down.
I20260619 10:19:53.596589 16642 raft_consensus.cc:2258] T 20be40cde00c432cbd3abb7e3fd762a8 P 69870ff4ad6a44ecb692fb40428dec06 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 10:19:53.597015 16642 ts_tablet_manager.cc:1905] T 20be40cde00c432cbd3abb7e3fd762a8 P 69870ff4ad6a44ecb692fb40428dec06: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 10:19:53.598030 16641 ts_tablet_manager.cc:1918] T 20be40cde00c432cbd3abb7e3fd762a8 P c1b395dd05a64924a62063d847ac8bb2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 10:19:53.598089 16641 log.cc:1196] T 20be40cde00c432cbd3abb7e3fd762a8 P c1b395dd05a64924a62063d847ac8bb2: Deleting WAL directory at /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781864224577058-825-0/minicluster-data/ts-2/wal/wals/20be40cde00c432cbd3abb7e3fd762a8
I20260619 10:19:53.598362 16641 ts_tablet_manager.cc:1939] T 20be40cde00c432cbd3abb7e3fd762a8 P c1b395dd05a64924a62063d847ac8bb2: Deleting consensus metadata
I20260619 10:19:53.598763 16131 catalog_manager.cc:4841] TS c1b395dd05a64924a62063d847ac8bb2 (127.0.206.67:38085): tablet 20be40cde00c432cbd3abb7e3fd762a8 (table test-table [id=8381967a37ca41fd9f78949ad371851e]) successfully deleted
I20260619 10:19:53.599026 16642 ts_tablet_manager.cc:1918] T 20be40cde00c432cbd3abb7e3fd762a8 P 69870ff4ad6a44ecb692fb40428dec06: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 10:19:53.599084 16642 log.cc:1196] T 20be40cde00c432cbd3abb7e3fd762a8 P 69870ff4ad6a44ecb692fb40428dec06: Deleting WAL directory at /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781864224577058-825-0/minicluster-data/ts-1/wal/wals/20be40cde00c432cbd3abb7e3fd762a8
I20260619 10:19:53.599321 16642 ts_tablet_manager.cc:1939] T 20be40cde00c432cbd3abb7e3fd762a8 P 69870ff4ad6a44ecb692fb40428dec06: Deleting consensus metadata
I20260619 10:19:53.599828 16129 catalog_manager.cc:4841] TS 69870ff4ad6a44ecb692fb40428dec06 (127.0.206.66:34349): tablet 20be40cde00c432cbd3abb7e3fd762a8 (table test-table [id=8381967a37ca41fd9f78949ad371851e]) successfully deleted
W20260619 10:19:53.602691 16128 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.0.206.65:41351 (error 108)
I20260619 10:19:53.603109 16613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:19:53.602987 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.65:41351 (local address 127.0.0.1:36114)
0619 10:19:53.603039 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0619 10:19:53.603072 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.65:41351: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260619 10:19:53.603219 16128 catalog_manager.cc:4584] TS d835bbc06b3d47f7b10e69f31a298ed9 (127.0.206.65:41351): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 20be40cde00c432cbd3abb7e3fd762a8: Network error: Client connection negotiation failed: client connection to 127.0.206.65:41351: connect: Connection refused (error 111)
I20260619 10:19:53.603518 825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTU9OP6/build/release/bin/kudu with pid 16340
I20260619 10:19:53.609529 825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTU9OP6/build/release/bin/kudu with pid 16474
I20260619 10:19:53.613777 825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTU9OP6/build/release/bin/kudu with pid 16112
I20260619 10:20:13.618916 825 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTU9OP6/build/release/bin/kudu
/tmp/dist-test-taskTU9OP6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781864224577058-825-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781864224577058-825-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781864224577058-825-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781864224577058-825-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.206.126:33511
--webserver_interface=127.0.206.126
--webserver_port=43223
--keytab_file=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781864224577058-825-0/krb5kdc/kudu_127.0.206.126.keytab
--principal=kudu/127.0.206.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.206.84:43867
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.206.126:33511
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781864224577058-825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781864224577058-825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781864224577058-825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 10:20:13.699483 16644 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 10:20:13.699698 16644 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 10:20:13.699719 16644 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 10:20:13.700835 16644 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260619 10:20:13.700871 16644 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260619 10:20:13.700886 16644 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260619 10:20:13.700927 16644 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 10:20:13.700946 16644 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 10:20:13.700970 16644 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 10:20:13.700995 16644 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 10:20:13.701023 16644 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 10:20:13.702513 16644 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.206.84:43867
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781864224577058-825-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781864224577058-825-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.0.206.126:33511
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781864224577058-825-0/krb5kdc/kudu_127.0.206.126.keytab
--principal=kudu/127.0.206.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.206.126:33511
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781864224577058-825-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.206.126
--webserver_port=43223
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16644
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781864224577058-825-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 09:32:44 UTC on ccycloud.quasar-gbtldp.root.comops.site
build id 4455440
I20260619 10:20:13.702768 16644 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 10:20:13.703012 16644 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 10:20:13.711308 16653 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 10:20:13.711373 16651 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 10:20:13.711308 16650 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 10:20:13.711426 16644 server_base.cc:1034] running on GCE node
I20260619 10:20:13.711831 16644 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 10:20:13.712142 16644 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 10:20:13.713292 16644 hybrid_clock.cc:648] HybridClock initialized: now 1781864413713284 us; error 47 us; skew 500 ppm
Jun 19 10:20:13 dist-test-slave-6pt2 krb5kdc[16099](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864413, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 10:20:13.739315 16644 init.cc:377] Logged in from keytab as kudu/127.0.206.126@KRBTEST.COM (short username kudu)
I20260619 10:20:13.739768 16644 webserver.cc:460] Webserver started at http://127.0.206.126:43223/ using document root <none> and password file <none>
I20260619 10:20:13.740022 16644 fs_manager.cc:359] Metadata directory not provided
I20260619 10:20:13.740111 16644 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 10:20:13.741459 16644 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260619 10:20:13.742067 16660 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 10:20:13.742241 16644 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260619 10:20:13.742341 16644 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781864224577058-825-0/minicluster-data/master-0/data,/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781864224577058-825-0/minicluster-data/master-0/wal
uuid: "b4330665f37e4f6693f0900bed7dcfe6"
format_stamp: "Formatted at 2026-06-19 10:19:52 on dist-test-slave-6pt2"
server_key: "1b36e4882d0518269926abeae9ba3acc"
server_key_iv: "8dd3ce24ed149b1d6aab6338fcf9b6df"
server_key_version: "encryptionkey@0"
I20260619 10:20:13.742805 16644 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781864224577058-825-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781864224577058-825-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781864224577058-825-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 10:20:13.782970 16644 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 10:20:13.785211 16644 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 10:20:13.785383 16644 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 10:20:13.789371 16644 rpc_server.cc:307] RPC server started. Bound to: 127.0.206.126:33511
I20260619 10:20:13.789403 16712 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.206.126:33511 every 8 connection(s)
I20260619 10:20:13.789821 16644 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781864224577058-825-0/minicluster-data/master-0/data/info.pb
I20260619 10:20:13.791333 16713 sys_catalog.cc:263] Verifying existing consensus state
I20260619 10:20:13.791947 16713 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b4330665f37e4f6693f0900bed7dcfe6: Bootstrap starting.
I20260619 10:20:13.794350 825 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTU9OP6/build/release/bin/kudu as pid 16644
I20260619 10:20:13.794737 825 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTU9OP6/build/release/bin/kudu
/tmp/dist-test-taskTU9OP6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781864224577058-825-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781864224577058-825-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781864224577058-825-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781864224577058-825-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.206.65:41351
--local_ip_for_outbound_sockets=127.0.206.65
--tserver_master_addrs=127.0.206.126:33511
--webserver_port=38183
--webserver_interface=127.0.206.65
--keytab_file=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781864224577058-825-0/krb5kdc/kudu_127.0.206.65.keytab
--principal=kudu/127.0.206.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.206.84:43867
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781864224577058-825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781864224577058-825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781864224577058-825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 10:20:13.800662 16713 log.cc:824] T 00000000000000000000000000000000 P b4330665f37e4f6693f0900bed7dcfe6: Log is configured to *not* fsync() on all Append() calls
I20260619 10:20:13.802507 16713 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b4330665f37e4f6693f0900bed7dcfe6: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260619 10:20:13.802822 16713 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b4330665f37e4f6693f0900bed7dcfe6: Bootstrap complete.
I20260619 10:20:13.804970 16713 raft_consensus.cc:348] T 00000000000000000000000000000000 P b4330665f37e4f6693f0900bed7dcfe6 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b4330665f37e4f6693f0900bed7dcfe6" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 33511 } }
I20260619 10:20:13.805177 16713 raft_consensus.cc:729] T 00000000000000000000000000000000 P b4330665f37e4f6693f0900bed7dcfe6 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: b4330665f37e4f6693f0900bed7dcfe6, State: Initialized, Role: FOLLOWER
I20260619 10:20:13.805339 16713 consensus_queue.cc:260] T 00000000000000000000000000000000 P b4330665f37e4f6693f0900bed7dcfe6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b4330665f37e4f6693f0900bed7dcfe6" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 33511 } }
I20260619 10:20:13.805393 16713 raft_consensus.cc:388] T 00000000000000000000000000000000 P b4330665f37e4f6693f0900bed7dcfe6 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 10:20:13.805444 16713 raft_consensus.cc:482] T 00000000000000000000000000000000 P b4330665f37e4f6693f0900bed7dcfe6 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 10:20:13.805495 16713 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b4330665f37e4f6693f0900bed7dcfe6 [term 1 FOLLOWER]: Advancing to term 2
I20260619 10:20:13.806452 16713 raft_consensus.cc:504] T 00000000000000000000000000000000 P b4330665f37e4f6693f0900bed7dcfe6 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b4330665f37e4f6693f0900bed7dcfe6" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 33511 } }
I20260619 10:20:13.806533 16713 leader_election.cc:302] T 00000000000000000000000000000000 P b4330665f37e4f6693f0900bed7dcfe6 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b4330665f37e4f6693f0900bed7dcfe6; no voters:
I20260619 10:20:13.806733 16713 leader_election.cc:288] T 00000000000000000000000000000000 P b4330665f37e4f6693f0900bed7dcfe6 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260619 10:20:13.806766 16717 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b4330665f37e4f6693f0900bed7dcfe6 [term 2 FOLLOWER]: Leader election won for term 2
I20260619 10:20:13.806918 16717 raft_consensus.cc:686] T 00000000000000000000000000000000 P b4330665f37e4f6693f0900bed7dcfe6 [term 2 LEADER]: Becoming Leader. State: Replica: b4330665f37e4f6693f0900bed7dcfe6, State: Running, Role: LEADER
I20260619 10:20:13.807036 16713 sys_catalog.cc:564] T 00000000000000000000000000000000 P b4330665f37e4f6693f0900bed7dcfe6 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 10:20:13.807061 16717 consensus_queue.cc:237] T 00000000000000000000000000000000 P b4330665f37e4f6693f0900bed7dcfe6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b4330665f37e4f6693f0900bed7dcfe6" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 33511 } }
I20260619 10:20:13.807426 16718 sys_catalog.cc:455] T 00000000000000000000000000000000 P b4330665f37e4f6693f0900bed7dcfe6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "b4330665f37e4f6693f0900bed7dcfe6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b4330665f37e4f6693f0900bed7dcfe6" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 33511 } } }
I20260619 10:20:13.807438 16719 sys_catalog.cc:455] T 00000000000000000000000000000000 P b4330665f37e4f6693f0900bed7dcfe6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b4330665f37e4f6693f0900bed7dcfe6. Latest consensus state: current_term: 2 leader_uuid: "b4330665f37e4f6693f0900bed7dcfe6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b4330665f37e4f6693f0900bed7dcfe6" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 33511 } } }
I20260619 10:20:13.807602 16718 sys_catalog.cc:458] T 00000000000000000000000000000000 P b4330665f37e4f6693f0900bed7dcfe6 [sys.catalog]: This master's current role is: LEADER
I20260619 10:20:13.807612 16719 sys_catalog.cc:458] T 00000000000000000000000000000000 P b4330665f37e4f6693f0900bed7dcfe6 [sys.catalog]: This master's current role is: LEADER
I20260619 10:20:13.808118 16726 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 10:20:13.808990 16726 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 10:20:13.809085 16726 catalog_manager.cc:1259] Loaded cluster ID: a2afe9084c3b435aaa7c67b9feec0afd
I20260619 10:20:13.809118 16726 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 10:20:13.810345 16726 catalog_manager.cc:1504] Loading token signing keys...
I20260619 10:20:13.810513 16726 catalog_manager.cc:5879] T 00000000000000000000000000000000 P b4330665f37e4f6693f0900bed7dcfe6: Loaded TSK: 0
I20260619 10:20:13.820401 16726 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b4330665f37e4f6693f0900bed7dcfe6: Generated new TSK 1
I20260619 10:20:13.820482 16726 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260619 10:20:13.880239 16715 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 10:20:13.880439 16715 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 10:20:13.880462 16715 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 10:20:13.881596 16715 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 10:20:13.881634 16715 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 10:20:13.881660 16715 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.206.65
I20260619 10:20:13.883121 16715 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.206.84:43867
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781864224577058-825-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781864224577058-825-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781864224577058-825-0/krb5kdc/kudu_127.0.206.65.keytab
--principal=kudu/127.0.206.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.206.65:41351
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781864224577058-825-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.206.65
--webserver_port=38183
--tserver_master_addrs=127.0.206.126:33511
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16715
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.206.65
--log_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781864224577058-825-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 09:32:44 UTC on ccycloud.quasar-gbtldp.root.comops.site
build id 4455440
I20260619 10:20:13.883324 16715 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 10:20:13.883561 16715 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 10:20:13.889154 16741 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 10:20:13.889145 16740 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 10:20:13.889144 16743 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 10:20:13.889376 16715 server_base.cc:1034] running on GCE node
I20260619 10:20:13.889590 16715 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 10:20:13.889866 16715 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 10:20:13.891024 16715 hybrid_clock.cc:648] HybridClock initialized: now 1781864413891009 us; error 33 us; skew 500 ppm
Jun 19 10:20:13 dist-test-slave-6pt2 krb5kdc[16099](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864413, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 10:20:13.893461 16715 init.cc:377] Logged in from keytab as kudu/127.0.206.65@KRBTEST.COM (short username kudu)
I20260619 10:20:13.893883 16715 webserver.cc:460] Webserver started at http://127.0.206.65:38183/ using document root <none> and password file <none>
I20260619 10:20:13.894096 16715 fs_manager.cc:359] Metadata directory not provided
I20260619 10:20:13.894176 16715 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 10:20:13.895581 16715 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 10:20:13.896260 16750 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 10:20:13.896428 16715 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 10:20:13.896517 16715 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781864224577058-825-0/minicluster-data/ts-0/data,/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781864224577058-825-0/minicluster-data/ts-0/wal
uuid: "d835bbc06b3d47f7b10e69f31a298ed9"
format_stamp: "Formatted at 2026-06-19 10:19:53 on dist-test-slave-6pt2"
server_key: "96107af8396e60298dc6ab4457b9e42a"
server_key_iv: "8ffac846010f0255650acda1a7ead878"
server_key_version: "encryptionkey@0"
I20260619 10:20:13.896821 16715 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781864224577058-825-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781864224577058-825-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781864224577058-825-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 10:20:13.910140 16715 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 10:20:13.912362 16715 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 10:20:13.912510 16715 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 10:20:13.912716 16715 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 10:20:13.913179 16757 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260619 10:20:13.914565 16715 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260619 10:20:13.914664 16715 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260619 10:20:13.914716 16715 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260619 10:20:13.915354 16715 ts_tablet_manager.cc:610] Registered 1 tablets
I20260619 10:20:13.915398 16715 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260619 10:20:13.915436 16757 tablet_bootstrap.cc:492] T 20be40cde00c432cbd3abb7e3fd762a8 P d835bbc06b3d47f7b10e69f31a298ed9: Bootstrap starting.
I20260619 10:20:13.923062 16715 rpc_server.cc:307] RPC server started. Bound to: 127.0.206.65:41351
I20260619 10:20:13.923516 16715 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781864224577058-825-0/minicluster-data/ts-0/data/info.pb
I20260619 10:20:13.927040 16757 log.cc:824] T 20be40cde00c432cbd3abb7e3fd762a8 P d835bbc06b3d47f7b10e69f31a298ed9: Log is configured to *not* fsync() on all Append() calls
I20260619 10:20:13.927338 16864 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.206.65:41351 every 8 connection(s)
I20260619 10:20:13.929451 16757 tablet_bootstrap.cc:492] T 20be40cde00c432cbd3abb7e3fd762a8 P d835bbc06b3d47f7b10e69f31a298ed9: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260619 10:20:13.929848 16757 tablet_bootstrap.cc:492] T 20be40cde00c432cbd3abb7e3fd762a8 P d835bbc06b3d47f7b10e69f31a298ed9: Bootstrap complete.
I20260619 10:20:13.931097 16757 ts_tablet_manager.cc:1397] T 20be40cde00c432cbd3abb7e3fd762a8 P d835bbc06b3d47f7b10e69f31a298ed9: Time spent bootstrapping tablet: real 0.016s user 0.003s sys 0.012s
I20260619 10:20:13.931586 825 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTU9OP6/build/release/bin/kudu as pid 16715
I20260619 10:20:13.931872 825 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTU9OP6/build/release/bin/kudu
/tmp/dist-test-taskTU9OP6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781864224577058-825-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781864224577058-825-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781864224577058-825-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781864224577058-825-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.206.66:34349
--local_ip_for_outbound_sockets=127.0.206.66
--tserver_master_addrs=127.0.206.126:33511
--webserver_port=39951
--webserver_interface=127.0.206.66
--keytab_file=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781864224577058-825-0/krb5kdc/kudu_127.0.206.66.keytab
--principal=kudu/127.0.206.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.206.84:43867
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781864224577058-825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781864224577058-825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781864224577058-825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Jun 19 10:20:13 dist-test-slave-6pt2 krb5kdc[16099](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864413, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.65@KRBTEST.COM for kudu/127.0.206.126@KRBTEST.COM
I20260619 10:20:13.932585 16757 raft_consensus.cc:348] T 20be40cde00c432cbd3abb7e3fd762a8 P d835bbc06b3d47f7b10e69f31a298ed9 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d835bbc06b3d47f7b10e69f31a298ed9" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 41351 } } peers { permanent_uuid: "69870ff4ad6a44ecb692fb40428dec06" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 34349 } } peers { permanent_uuid: "c1b395dd05a64924a62063d847ac8bb2" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 38085 } }
I20260619 10:20:13.933033 16757 raft_consensus.cc:729] T 20be40cde00c432cbd3abb7e3fd762a8 P d835bbc06b3d47f7b10e69f31a298ed9 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: d835bbc06b3d47f7b10e69f31a298ed9, State: Initialized, Role: FOLLOWER
I20260619 10:20:13.933264 16757 consensus_queue.cc:260] T 20be40cde00c432cbd3abb7e3fd762a8 P d835bbc06b3d47f7b10e69f31a298ed9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d835bbc06b3d47f7b10e69f31a298ed9" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 41351 } } peers { permanent_uuid: "69870ff4ad6a44ecb692fb40428dec06" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 34349 } } peers { permanent_uuid: "c1b395dd05a64924a62063d847ac8bb2" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 38085 } }
I20260619 10:20:13.933537 16757 ts_tablet_manager.cc:1428] T 20be40cde00c432cbd3abb7e3fd762a8 P d835bbc06b3d47f7b10e69f31a298ed9: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260619 10:20:13.936470 16867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:20:13.924360 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:33511 (local address 127.0.206.65:48925)
0619 10:20:13.924542 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0619 10:20:13.924607 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0619 10:20:13.925347 (+ 740us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 10:20:13.925838 (+ 491us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 10:20:13.925852 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 10:20:13.926320 (+ 468us) client_negotiation.cc:189] Negotiated authn=SASL
0619 10:20:13.926642 (+ 322us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:20:13.926649 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:20:13.928274 (+ 1625us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:20:13.928275 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:20:13.929473 (+ 1198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:20:13.929476 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:20:13.930385 (+ 909us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:20:13.930387 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:20:13.930523 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:20:13.931166 (+ 643us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 10:20:13.931176 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0619 10:20:13.932892 (+ 1716us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 10:20:13.934938 (+ 2046us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 10:20:13.934944 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 10:20:13.934946 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 10:20:13.935153 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 10:20:13.935413 (+ 260us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 10:20:13.935413 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 10:20:13.935414 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 10:20:13.935516 (+ 102us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 10:20:13.935877 (+ 361us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 10:20:13.935883 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 10:20:13.936019 (+ 136us) client_negotiation.cc:769] Sending connection context
0619 10:20:13.936117 (+ 98us) client_negotiation.cc:240] Negotiation successful
0619 10:20:13.936277 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":92,"threads_started":1}
I20260619 10:20:13.936468 16868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:20:13.924612 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.65:48925 (local address 127.0.206.126:33511)
0619 10:20:13.924766 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0619 10:20:13.924775 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 10:20:13.924807 (+ 32us) server_negotiation.cc:407] Connection header received
0619 10:20:13.925581 (+ 774us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:20:13.925587 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:20:13.925639 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:20:13.925772 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:20:13.926779 (+ 1007us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:20:13.928145 (+ 1366us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:20:13.929575 (+ 1430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:20:13.930263 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:20:13.930380 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:20:13.933005 (+ 2625us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 10:20:13.933008 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 10:20:13.933009 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 10:20:13.933025 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0619 10:20:13.934849 (+ 1824us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:20:13.935247 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:20:13.935250 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:20:13.935251 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:20:13.935335 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:20:13.935580 (+ 245us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:20:13.935597 (+ 17us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:20:13.935598 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:20:13.935809 (+ 211us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 10:20:13.935898 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0619 10:20:13.936166 (+ 268us) server_negotiation.cc:299] Negotiation successful
0619 10:20:13.936274 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":117,"thread_start_us":62,"threads_started":1}
I20260619 10:20:13.937239 16865 heartbeater.cc:347] Connected to a master server at 127.0.206.126:33511
I20260619 10:20:13.937333 16865 heartbeater.cc:464] Registering TS with master...
I20260619 10:20:13.937512 16865 heartbeater.cc:511] Master 127.0.206.126:33511 requested a full tablet report, sending...
I20260619 10:20:13.938235 16677 ts_manager.cc:194] Registered new tserver with Master: d835bbc06b3d47f7b10e69f31a298ed9 (127.0.206.65:41351)
I20260619 10:20:13.939527 16677 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.206.65@KRBTEST.COM'} at 127.0.206.65:48925
I20260619 10:20:13.940495 16865 heartbeater.cc:503] Master 127.0.206.126:33511 was elected leader, sending a full tablet report...
Jun 19 10:20:13 dist-test-slave-6pt2 krb5kdc[16099](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864413, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.126@KRBTEST.COM for kudu/127.0.206.65@KRBTEST.COM
I20260619 10:20:13.947497 16872 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:20:13.938968 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.65:41351 (local address 127.0.0.1:58404)
0619 10:20:13.939123 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0619 10:20:13.939153 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0619 10:20:13.939213 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 10:20:13.940055 (+ 842us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 10:20:13.940060 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 10:20:13.940359 (+ 299us) client_negotiation.cc:189] Negotiated authn=SASL
0619 10:20:13.940407 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:20:13.940410 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:20:13.941709 (+ 1299us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:20:13.941709 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:20:13.942249 (+ 540us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:20:13.942251 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:20:13.942849 (+ 598us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:20:13.942850 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:20:13.942899 (+ 49us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:20:13.943342 (+ 443us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 10:20:13.943346 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 10:20:13.944551 (+ 1205us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 10:20:13.946415 (+ 1864us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 10:20:13.946421 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 10:20:13.946422 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 10:20:13.946641 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 10:20:13.946839 (+ 198us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 10:20:13.946840 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 10:20:13.946841 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 10:20:13.946894 (+ 53us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 10:20:13.947168 (+ 274us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 10:20:13.947173 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 10:20:13.947217 (+ 44us) client_negotiation.cc:769] Sending connection context
0619 10:20:13.947309 (+ 92us) client_negotiation.cc:240] Negotiation successful
0619 10:20:13.947412 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":104,"threads_started":1}
I20260619 10:20:13.947497 16873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:20:13.939016 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58404 (local address 127.0.206.65:41351)
0619 10:20:13.939886 (+ 870us) server_negotiation.cc:206] Beginning negotiation
0619 10:20:13.939891 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 10:20:13.939903 (+ 12us) server_negotiation.cc:407] Connection header received
0619 10:20:13.939937 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:20:13.939939 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:20:13.939958 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:20:13.940042 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:20:13.940500 (+ 458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:20:13.941632 (+ 1132us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:20:13.942333 (+ 701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:20:13.942780 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:20:13.942856 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:20:13.944623 (+ 1767us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 10:20:13.944630 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 10:20:13.944630 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 10:20:13.944637 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 10:20:13.946347 (+ 1710us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:20:13.946712 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:20:13.946714 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:20:13.946715 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:20:13.946778 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:20:13.946962 (+ 184us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:20:13.946962 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:20:13.946963 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:20:13.947089 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 10:20:13.947173 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0619 10:20:13.947342 (+ 169us) server_negotiation.cc:299] Negotiation successful
0619 10:20:13.947419 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":837,"thread_start_us":70,"threads_started":1}
I20260619 10:20:13.947857 16799 tablet_service.cc:1514] Processing DeleteTablet for tablet 20be40cde00c432cbd3abb7e3fd762a8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 10:19:53 UTC) from {username='kudu', principal='kudu/127.0.206.126@KRBTEST.COM'} at 127.0.0.1:58404
I20260619 10:20:13.948289 16874 tablet_replica.cc:330] stopping tablet replica
I20260619 10:20:13.948390 16874 raft_consensus.cc:2229] T 20be40cde00c432cbd3abb7e3fd762a8 P d835bbc06b3d47f7b10e69f31a298ed9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 10:20:13.948451 16874 raft_consensus.cc:2258] T 20be40cde00c432cbd3abb7e3fd762a8 P d835bbc06b3d47f7b10e69f31a298ed9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 10:20:13.948734 16874 ts_tablet_manager.cc:1905] T 20be40cde00c432cbd3abb7e3fd762a8 P d835bbc06b3d47f7b10e69f31a298ed9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 10:20:13.951016 16874 ts_tablet_manager.cc:1918] T 20be40cde00c432cbd3abb7e3fd762a8 P d835bbc06b3d47f7b10e69f31a298ed9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 10:20:13.951089 16874 log.cc:1196] T 20be40cde00c432cbd3abb7e3fd762a8 P d835bbc06b3d47f7b10e69f31a298ed9: Deleting WAL directory at /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781864224577058-825-0/minicluster-data/ts-0/wal/wals/20be40cde00c432cbd3abb7e3fd762a8
I20260619 10:20:13.951402 16874 ts_tablet_manager.cc:1939] T 20be40cde00c432cbd3abb7e3fd762a8 P d835bbc06b3d47f7b10e69f31a298ed9: Deleting consensus metadata
I20260619 10:20:13.951872 16661 catalog_manager.cc:4841] TS d835bbc06b3d47f7b10e69f31a298ed9 (127.0.206.65:41351): tablet 20be40cde00c432cbd3abb7e3fd762a8 (table test-table [id=8381967a37ca41fd9f78949ad371851e]) successfully deleted
W20260619 10:20:14.014068 16870 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 10:20:14.014246 16870 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 10:20:14.014268 16870 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 10:20:14.015391 16870 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 10:20:14.015429 16870 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 10:20:14.015482 16870 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.206.66
I20260619 10:20:14.016903 16870 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.206.84:43867
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781864224577058-825-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781864224577058-825-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781864224577058-825-0/krb5kdc/kudu_127.0.206.66.keytab
--principal=kudu/127.0.206.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.206.66:34349
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781864224577058-825-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.206.66
--webserver_port=39951
--tserver_master_addrs=127.0.206.126:33511
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16870
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.206.66
--log_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781864224577058-825-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 09:32:44 UTC on ccycloud.quasar-gbtldp.root.comops.site
build id 4455440
I20260619 10:20:14.017134 16870 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 10:20:14.017378 16870 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 10:20:14.022099 16882 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 10:20:14.022130 16880 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 10:20:14.022181 16879 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 10:20:14.022225 16870 server_base.cc:1034] running on GCE node
I20260619 10:20:14.022506 16870 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 10:20:14.022770 16870 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 10:20:14.023938 16870 hybrid_clock.cc:648] HybridClock initialized: now 1781864414023927 us; error 27 us; skew 500 ppm
Jun 19 10:20:14 dist-test-slave-6pt2 krb5kdc[16099](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864414, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 10:20:14.026427 16870 init.cc:377] Logged in from keytab as kudu/127.0.206.66@KRBTEST.COM (short username kudu)
I20260619 10:20:14.026887 16870 webserver.cc:460] Webserver started at http://127.0.206.66:39951/ using document root <none> and password file <none>
I20260619 10:20:14.027112 16870 fs_manager.cc:359] Metadata directory not provided
I20260619 10:20:14.027220 16870 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 10:20:14.028448 16870 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260619 10:20:14.029008 16889 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 10:20:14.029171 16870 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260619 10:20:14.029265 16870 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781864224577058-825-0/minicluster-data/ts-1/data,/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781864224577058-825-0/minicluster-data/ts-1/wal
uuid: "69870ff4ad6a44ecb692fb40428dec06"
format_stamp: "Formatted at 2026-06-19 10:19:53 on dist-test-slave-6pt2"
server_key: "47ac82c46eba2f3daf1ebc6486ee0b63"
server_key_iv: "76cc672c9e34ed3da1f5b39f9c85b5fd"
server_key_version: "encryptionkey@0"
I20260619 10:20:14.029597 16870 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781864224577058-825-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781864224577058-825-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781864224577058-825-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 10:20:14.064083 16870 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 10:20:14.066321 16870 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 10:20:14.066483 16870 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 10:20:14.066776 16870 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 10:20:14.067099 16870 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 10:20:14.067159 16870 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 10:20:14.067195 16870 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 10:20:14.067222 16870 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 10:20:14.072618 16870 rpc_server.cc:307] RPC server started. Bound to: 127.0.206.66:34349
I20260619 10:20:14.072703 17002 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.206.66:34349 every 8 connection(s)
I20260619 10:20:14.073014 16870 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781864224577058-825-0/minicluster-data/ts-1/data/info.pb
I20260619 10:20:14.077232 825 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTU9OP6/build/release/bin/kudu as pid 16870
I20260619 10:20:14.077538 825 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTU9OP6/build/release/bin/kudu
/tmp/dist-test-taskTU9OP6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781864224577058-825-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781864224577058-825-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781864224577058-825-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781864224577058-825-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.206.67:38085
--local_ip_for_outbound_sockets=127.0.206.67
--tserver_master_addrs=127.0.206.126:33511
--webserver_port=33907
--webserver_interface=127.0.206.67
--keytab_file=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781864224577058-825-0/krb5kdc/kudu_127.0.206.67.keytab
--principal=kudu/127.0.206.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.206.84:43867
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781864224577058-825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781864224577058-825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781864224577058-825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Jun 19 10:20:14 dist-test-slave-6pt2 krb5kdc[16099](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864414, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.66@KRBTEST.COM for kudu/127.0.206.126@KRBTEST.COM
I20260619 10:20:14.083475 17005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:20:14.073716 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:33511 (local address 127.0.206.66:58217)
0619 10:20:14.073882 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0619 10:20:14.073914 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0619 10:20:14.074408 (+ 494us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 10:20:14.074674 (+ 266us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 10:20:14.074679 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 10:20:14.075247 (+ 568us) client_negotiation.cc:189] Negotiated authn=SASL
0619 10:20:14.075474 (+ 227us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:20:14.075479 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:20:14.076252 (+ 773us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:20:14.076253 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:20:14.077159 (+ 906us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:20:14.077162 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:20:14.077654 (+ 492us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:20:14.077656 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:20:14.077804 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:20:14.078276 (+ 472us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 10:20:14.078288 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0619 10:20:14.080181 (+ 1893us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 10:20:14.082119 (+ 1938us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 10:20:14.082124 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 10:20:14.082126 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 10:20:14.082341 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 10:20:14.082567 (+ 226us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 10:20:14.082568 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 10:20:14.082568 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 10:20:14.082720 (+ 152us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 10:20:14.083028 (+ 308us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 10:20:14.083032 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 10:20:14.083139 (+ 107us) client_negotiation.cc:769] Sending connection context
0619 10:20:14.083209 (+ 70us) client_negotiation.cc:240] Negotiation successful
0619 10:20:14.083321 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":77,"threads_started":1}
I20260619 10:20:14.083513 16868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:20:14.073719 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.66:58217 (local address 127.0.206.126:33511)
0619 10:20:14.073765 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0619 10:20:14.073766 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 10:20:14.074000 (+ 234us) server_negotiation.cc:407] Connection header received
0619 10:20:14.074515 (+ 515us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:20:14.074516 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:20:14.074529 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:20:14.074576 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:20:14.075612 (+ 1036us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:20:14.076145 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:20:14.077258 (+ 1113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:20:14.077539 (+ 281us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:20:14.077636 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:20:14.080308 (+ 2672us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 10:20:14.080309 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 10:20:14.080310 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 10:20:14.080318 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 10:20:14.082022 (+ 1704us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:20:14.082416 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:20:14.082417 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:20:14.082418 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:20:14.082484 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:20:14.082786 (+ 302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:20:14.082787 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:20:14.082788 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:20:14.082954 (+ 166us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 10:20:14.083028 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0619 10:20:14.083276 (+ 248us) server_negotiation.cc:299] Negotiation successful
0619 10:20:14.083372 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260619 10:20:14.084117 17003 heartbeater.cc:347] Connected to a master server at 127.0.206.126:33511
I20260619 10:20:14.084220 17003 heartbeater.cc:464] Registering TS with master...
I20260619 10:20:14.084408 17003 heartbeater.cc:511] Master 127.0.206.126:33511 requested a full tablet report, sending...
I20260619 10:20:14.084823 16677 ts_manager.cc:194] Registered new tserver with Master: 69870ff4ad6a44ecb692fb40428dec06 (127.0.206.66:34349)
I20260619 10:20:14.085352 16677 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.206.66@KRBTEST.COM'} at 127.0.206.66:58217
W20260619 10:20:14.157871 17006 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 10:20:14.158056 17006 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 10:20:14.158078 17006 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 10:20:14.159162 17006 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 10:20:14.159195 17006 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 10:20:14.159219 17006 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.206.67
I20260619 10:20:14.160558 17006 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.206.84:43867
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781864224577058-825-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781864224577058-825-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781864224577058-825-0/krb5kdc/kudu_127.0.206.67.keytab
--principal=kudu/127.0.206.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.206.67:38085
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781864224577058-825-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.206.67
--webserver_port=33907
--tserver_master_addrs=127.0.206.126:33511
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17006
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.206.67
--log_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781864224577058-825-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 09:32:44 UTC on ccycloud.quasar-gbtldp.root.comops.site
build id 4455440
I20260619 10:20:14.160737 17006 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 10:20:14.160894 17006 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260619 10:20:14.165766 17006 server_base.cc:1034] running on GCE node
W20260619 10:20:14.165714 17012 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 10:20:14.165720 17011 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 10:20:14.165714 17014 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 10:20:14.166081 17006 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 10:20:14.166332 17006 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 10:20:14.167488 17006 hybrid_clock.cc:648] HybridClock initialized: now 1781864414167474 us; error 27 us; skew 500 ppm
Jun 19 10:20:14 dist-test-slave-6pt2 krb5kdc[16099](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864414, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 10:20:14.169806 17006 init.cc:377] Logged in from keytab as kudu/127.0.206.67@KRBTEST.COM (short username kudu)
I20260619 10:20:14.170171 17006 webserver.cc:460] Webserver started at http://127.0.206.67:33907/ using document root <none> and password file <none>
I20260619 10:20:14.170358 17006 fs_manager.cc:359] Metadata directory not provided
I20260619 10:20:14.170409 17006 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 10:20:14.171671 17006 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 10:20:14.172183 17021 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 10:20:14.172333 17006 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 10:20:14.172421 17006 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781864224577058-825-0/minicluster-data/ts-2/data,/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781864224577058-825-0/minicluster-data/ts-2/wal
uuid: "c1b395dd05a64924a62063d847ac8bb2"
format_stamp: "Formatted at 2026-06-19 10:19:53 on dist-test-slave-6pt2"
server_key: "b86270f85a9aa898f56881c27592156e"
server_key_iv: "e0fdc184a4f5eb447f166ff615c876b4"
server_key_version: "encryptionkey@0"
I20260619 10:20:14.172724 17006 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781864224577058-825-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781864224577058-825-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781864224577058-825-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 10:20:14.182567 17006 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 10:20:14.185148 17006 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 10:20:14.185302 17006 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 10:20:14.185523 17006 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 10:20:14.185811 17006 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 10:20:14.185858 17006 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 10:20:14.185879 17006 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 10:20:14.185923 17006 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 10:20:14.191612 17006 rpc_server.cc:307] RPC server started. Bound to: 127.0.206.67:38085
I20260619 10:20:14.191695 17134 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.206.67:38085 every 8 connection(s)
I20260619 10:20:14.192013 17006 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781864224577058-825-0/minicluster-data/ts-2/data/info.pb
I20260619 10:20:14.193164 825 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTU9OP6/build/release/bin/kudu as pid 17006
Jun 19 10:20:14 dist-test-slave-6pt2 krb5kdc[16099](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864414, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.67@KRBTEST.COM for kudu/127.0.206.126@KRBTEST.COM
I20260619 10:20:14.200450 17139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:20:14.193568 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47872 (local address 127.0.206.126:33511)
0619 10:20:14.193687 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0619 10:20:14.193688 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 10:20:14.193903 (+ 215us) server_negotiation.cc:407] Connection header received
0619 10:20:14.193976 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:20:14.193977 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:20:14.193988 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:20:14.194045 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:20:14.194698 (+ 653us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:20:14.195154 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:20:14.195891 (+ 737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:20:14.196169 (+ 278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:20:14.196256 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:20:14.197522 (+ 1266us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 10:20:14.197524 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 10:20:14.197524 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 10:20:14.197531 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 10:20:14.199147 (+ 1616us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:20:14.199574 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:20:14.199575 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:20:14.199576 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:20:14.199630 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:20:14.199870 (+ 240us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:20:14.199871 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:20:14.199872 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:20:14.200023 (+ 151us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 10:20:14.200198 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0619 10:20:14.200236 (+ 38us) server_negotiation.cc:299] Negotiation successful
0619 10:20:14.200326 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":101,"thread_start_us":70,"threads_started":1}
I20260619 10:20:14.201938 16868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:20:14.192763 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.67:47525 (local address 127.0.206.126:33511)
0619 10:20:14.192817 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0619 10:20:14.192819 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 10:20:14.193011 (+ 192us) server_negotiation.cc:407] Connection header received
0619 10:20:14.193619 (+ 608us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:20:14.193620 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:20:14.193630 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:20:14.193704 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:20:14.194668 (+ 964us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:20:14.195145 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:20:14.196203 (+ 1058us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:20:14.196511 (+ 308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:20:14.196575 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:20:14.198956 (+ 2381us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 10:20:14.198957 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 10:20:14.198958 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 10:20:14.198964 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 10:20:14.200579 (+ 1615us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:20:14.201030 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:20:14.201031 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:20:14.201031 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:20:14.201083 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:20:14.201340 (+ 257us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:20:14.201341 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:20:14.201342 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:20:14.201442 (+ 100us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 10:20:14.201513 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0619 10:20:14.201778 (+ 265us) server_negotiation.cc:299] Negotiation successful
0619 10:20:14.201853 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260619 10:20:14.202139 17137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:20:14.192757 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:33511 (local address 127.0.206.67:47525)
0619 10:20:14.192895 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0619 10:20:14.192927 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0619 10:20:14.193521 (+ 594us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 10:20:14.193786 (+ 265us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 10:20:14.193790 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 10:20:14.194350 (+ 560us) client_negotiation.cc:189] Negotiated authn=SASL
0619 10:20:14.194525 (+ 175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:20:14.194529 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:20:14.195233 (+ 704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:20:14.195234 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:20:14.196103 (+ 869us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:20:14.196105 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:20:14.196593 (+ 488us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:20:14.196594 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:20:14.196752 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:20:14.197222 (+ 470us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 10:20:14.197229 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0619 10:20:14.198849 (+ 1620us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 10:20:14.200660 (+ 1811us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 10:20:14.200666 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 10:20:14.200668 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 10:20:14.200953 (+ 285us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 10:20:14.201155 (+ 202us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 10:20:14.201155 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 10:20:14.201156 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 10:20:14.201268 (+ 112us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 10:20:14.201507 (+ 239us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 10:20:14.201512 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 10:20:14.201643 (+ 131us) client_negotiation.cc:769] Sending connection context
0619 10:20:14.201741 (+ 98us) client_negotiation.cc:240] Negotiation successful
0619 10:20:14.201904 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":81,"threads_started":1}
I20260619 10:20:14.202576 17135 heartbeater.cc:347] Connected to a master server at 127.0.206.126:33511
I20260619 10:20:14.202697 17135 heartbeater.cc:464] Registering TS with master...
I20260619 10:20:14.202831 17135 heartbeater.cc:511] Master 127.0.206.126:33511 requested a full tablet report, sending...
I20260619 10:20:14.203138 16677 ts_manager.cc:194] Registered new tserver with Master: c1b395dd05a64924a62063d847ac8bb2 (127.0.206.67:38085)
I20260619 10:20:14.203522 16677 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.206.67@KRBTEST.COM'} at 127.0.206.67:47525
I20260619 10:20:14.211850 825 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 10:20:14.214283 16868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:20:14.212158 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47882 (local address 127.0.206.126:33511)
0619 10:20:14.212226 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0619 10:20:14.212228 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 10:20:14.212317 (+ 89us) server_negotiation.cc:407] Connection header received
0619 10:20:14.212369 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:20:14.212374 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:20:14.212421 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:20:14.212486 (+ 65us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 10:20:14.212590 (+ 104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:20:14.213054 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:20:14.213724 (+ 670us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:20:14.213893 (+ 169us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:20:14.213937 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:20:14.214054 (+ 117us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 10:20:14.214089 (+ 35us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0619 10:20:14.214225 (+ 136us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:47882: token expired
Metrics: {"server-negotiator.queue_time_us":45}
W20260619 10:20:14.214336 16868 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:47882: token expired
W20260619 10:20:14.214356 16606 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.0.206.126:33511)
I20260619 10:20:14.219519 16868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:20:14.214565 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47886 (local address 127.0.206.126:33511)
0619 10:20:14.214634 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0619 10:20:14.214636 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 10:20:14.214647 (+ 11us) server_negotiation.cc:407] Connection header received
0619 10:20:14.214701 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:20:14.214702 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:20:14.214708 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:20:14.214747 (+ 39us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:20:14.215223 (+ 476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:20:14.215613 (+ 390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:20:14.216181 (+ 568us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:20:14.216351 (+ 170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:20:14.216394 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:20:14.217368 (+ 974us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 10:20:14.217369 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 10:20:14.217369 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 10:20:14.217375 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 10:20:14.218767 (+ 1392us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:20:14.219039 (+ 272us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:20:14.219040 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:20:14.219040 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:20:14.219084 (+ 44us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:20:14.219189 (+ 105us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:20:14.219190 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:20:14.219190 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:20:14.219264 (+ 74us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 10:20:14.219326 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0619 10:20:14.219391 (+ 65us) server_negotiation.cc:299] Negotiation successful
0619 10:20:14.219438 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260619 10:20:14.220504 16677 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47886:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 10:20:14.222018 16677 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 10:20:14.224234 16799 tablet_service.cc:1467] Processing CreateTablet for tablet ff30921c89da49ae958cbe5701678f1a (DEFAULT_TABLE table=test-table [id=ab9a01d8ddb943bb8e5d5246dd265fd8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 10:20:14.224437 16799 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ff30921c89da49ae958cbe5701678f1a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 10:20:14.226099 16757 tablet_bootstrap.cc:492] T ff30921c89da49ae958cbe5701678f1a P d835bbc06b3d47f7b10e69f31a298ed9: Bootstrap starting.
I20260619 10:20:14.226435 16757 tablet_bootstrap.cc:654] T ff30921c89da49ae958cbe5701678f1a P d835bbc06b3d47f7b10e69f31a298ed9: Neither blocks nor log segments found. Creating new log.
I20260619 10:20:14.227788 16757 tablet_bootstrap.cc:492] T ff30921c89da49ae958cbe5701678f1a P d835bbc06b3d47f7b10e69f31a298ed9: No bootstrap required, opened a new log
I20260619 10:20:14.227837 16757 ts_tablet_manager.cc:1397] T ff30921c89da49ae958cbe5701678f1a P d835bbc06b3d47f7b10e69f31a298ed9: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260619 10:20:14.227978 16757 raft_consensus.cc:348] T ff30921c89da49ae958cbe5701678f1a P d835bbc06b3d47f7b10e69f31a298ed9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69870ff4ad6a44ecb692fb40428dec06" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 34349 } } peers { permanent_uuid: "c1b395dd05a64924a62063d847ac8bb2" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 38085 } } peers { permanent_uuid: "d835bbc06b3d47f7b10e69f31a298ed9" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 41351 } }
I20260619 10:20:14.228057 16757 raft_consensus.cc:374] T ff30921c89da49ae958cbe5701678f1a P d835bbc06b3d47f7b10e69f31a298ed9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 10:20:14.228111 16757 raft_consensus.cc:729] T ff30921c89da49ae958cbe5701678f1a P d835bbc06b3d47f7b10e69f31a298ed9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d835bbc06b3d47f7b10e69f31a298ed9, State: Initialized, Role: FOLLOWER
I20260619 10:20:14.228160 16757 consensus_queue.cc:260] T ff30921c89da49ae958cbe5701678f1a P d835bbc06b3d47f7b10e69f31a298ed9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69870ff4ad6a44ecb692fb40428dec06" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 34349 } } peers { permanent_uuid: "c1b395dd05a64924a62063d847ac8bb2" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 38085 } } peers { permanent_uuid: "d835bbc06b3d47f7b10e69f31a298ed9" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 41351 } }
I20260619 10:20:14.228251 16757 ts_tablet_manager.cc:1428] T ff30921c89da49ae958cbe5701678f1a P d835bbc06b3d47f7b10e69f31a298ed9: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260619 10:20:14.230326 16872 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:20:14.223774 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.67:38085 (local address 127.0.0.1:48550)
0619 10:20:14.223803 (+ 29us) negotiation.cc:107] Waiting for socket to connect
0619 10:20:14.223812 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0619 10:20:14.223851 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 10:20:14.226734 (+ 2883us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 10:20:14.226735 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 10:20:14.226737 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 10:20:14.226786 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:20:14.226787 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:20:14.228085 (+ 1298us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:20:14.228087 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:20:14.229190 (+ 1103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:20:14.229193 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:20:14.230102 (+ 909us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:20:14.230102 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:20:14.230154 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:20:14.230167 (+ 13us) client_negotiation.cc:769] Sending connection context
0619 10:20:14.230212 (+ 45us) client_negotiation.cc:240] Negotiation successful
0619 10:20:14.230261 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":16}
I20260619 10:20:14.230356 17142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:20:14.226130 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48550 (local address 127.0.206.67:38085)
0619 10:20:14.226508 (+ 378us) server_negotiation.cc:206] Beginning negotiation
0619 10:20:14.226514 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 10:20:14.226528 (+ 14us) server_negotiation.cc:407] Connection header received
0619 10:20:14.226567 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:20:14.226569 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:20:14.226588 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:20:14.226868 (+ 280us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 10:20:14.226953 (+ 85us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:20:14.228002 (+ 1049us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:20:14.229266 (+ 1264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:20:14.230029 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:20:14.230098 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:20:14.230148 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0619 10:20:14.230226 (+ 78us) server_negotiation.cc:299] Negotiation successful
0619 10:20:14.230268 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":344,"thread_start_us":87,"threads_started":1}
I20260619 10:20:14.230985 17069 tablet_service.cc:1467] Processing CreateTablet for tablet ff30921c89da49ae958cbe5701678f1a (DEFAULT_TABLE table=test-table [id=ab9a01d8ddb943bb8e5d5246dd265fd8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 10:20:14.231290 17069 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ff30921c89da49ae958cbe5701678f1a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 10:20:14.231295 17143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:20:14.223780 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.66:34349 (local address 127.0.0.1:46994)
0619 10:20:14.227060 (+ 3280us) negotiation.cc:107] Waiting for socket to connect
0619 10:20:14.227071 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 10:20:14.227131 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 10:20:14.227495 (+ 364us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 10:20:14.227496 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 10:20:14.227498 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 10:20:14.227619 (+ 121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:20:14.227644 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:20:14.229140 (+ 1496us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:20:14.229140 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:20:14.230209 (+ 1069us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:20:14.230213 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:20:14.231077 (+ 864us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:20:14.231078 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:20:14.231136 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:20:14.231140 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 10:20:14.231196 (+ 56us) client_negotiation.cc:240] Negotiation successful
0619 10:20:14.231231 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3264,"mutex_wait_us":2560,"thread_start_us":76,"threads_started":1}
I20260619 10:20:14.231309 17141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:20:14.223908 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46994 (local address 127.0.206.66:34349)
0619 10:20:14.224124 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0619 10:20:14.224129 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 10:20:14.227150 (+ 3021us) server_negotiation.cc:407] Connection header received
0619 10:20:14.227225 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:20:14.227228 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:20:14.227267 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:20:14.227398 (+ 131us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 10:20:14.227742 (+ 344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:20:14.229062 (+ 1320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:20:14.230288 (+ 1226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:20:14.230992 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:20:14.231076 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:20:14.231124 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0619 10:20:14.231207 (+ 83us) server_negotiation.cc:299] Negotiation successful
0619 10:20:14.231240 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":150,"thread_start_us":83,"threads_started":1}
I20260619 10:20:14.233944 17145 tablet_bootstrap.cc:492] T ff30921c89da49ae958cbe5701678f1a P c1b395dd05a64924a62063d847ac8bb2: Bootstrap starting.
I20260619 10:20:14.234654 17145 tablet_bootstrap.cc:654] T ff30921c89da49ae958cbe5701678f1a P c1b395dd05a64924a62063d847ac8bb2: Neither blocks nor log segments found. Creating new log.
I20260619 10:20:14.234602 16937 tablet_service.cc:1467] Processing CreateTablet for tablet ff30921c89da49ae958cbe5701678f1a (DEFAULT_TABLE table=test-table [id=ab9a01d8ddb943bb8e5d5246dd265fd8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 10:20:14.234977 16937 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ff30921c89da49ae958cbe5701678f1a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 10:20:14.235255 17145 log.cc:824] T ff30921c89da49ae958cbe5701678f1a P c1b395dd05a64924a62063d847ac8bb2: Log is configured to *not* fsync() on all Append() calls
I20260619 10:20:14.235989 17145 tablet_bootstrap.cc:492] T ff30921c89da49ae958cbe5701678f1a P c1b395dd05a64924a62063d847ac8bb2: No bootstrap required, opened a new log
I20260619 10:20:14.236057 17145 ts_tablet_manager.cc:1397] T ff30921c89da49ae958cbe5701678f1a P c1b395dd05a64924a62063d847ac8bb2: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260619 10:20:14.238008 17145 raft_consensus.cc:348] T ff30921c89da49ae958cbe5701678f1a P c1b395dd05a64924a62063d847ac8bb2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69870ff4ad6a44ecb692fb40428dec06" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 34349 } } peers { permanent_uuid: "c1b395dd05a64924a62063d847ac8bb2" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 38085 } } peers { permanent_uuid: "d835bbc06b3d47f7b10e69f31a298ed9" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 41351 } }
I20260619 10:20:14.238183 17145 raft_consensus.cc:374] T ff30921c89da49ae958cbe5701678f1a P c1b395dd05a64924a62063d847ac8bb2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 10:20:14.238237 17145 raft_consensus.cc:729] T ff30921c89da49ae958cbe5701678f1a P c1b395dd05a64924a62063d847ac8bb2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c1b395dd05a64924a62063d847ac8bb2, State: Initialized, Role: FOLLOWER
I20260619 10:20:14.238363 17145 consensus_queue.cc:260] T ff30921c89da49ae958cbe5701678f1a P c1b395dd05a64924a62063d847ac8bb2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69870ff4ad6a44ecb692fb40428dec06" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 34349 } } peers { permanent_uuid: "c1b395dd05a64924a62063d847ac8bb2" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 38085 } } peers { permanent_uuid: "d835bbc06b3d47f7b10e69f31a298ed9" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 41351 } }
I20260619 10:20:14.238381 17147 tablet_bootstrap.cc:492] T ff30921c89da49ae958cbe5701678f1a P 69870ff4ad6a44ecb692fb40428dec06: Bootstrap starting.
I20260619 10:20:14.238638 17145 ts_tablet_manager.cc:1428] T ff30921c89da49ae958cbe5701678f1a P c1b395dd05a64924a62063d847ac8bb2: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260619 10:20:14.238667 17135 heartbeater.cc:503] Master 127.0.206.126:33511 was elected leader, sending a full tablet report...
I20260619 10:20:14.239028 17147 tablet_bootstrap.cc:654] T ff30921c89da49ae958cbe5701678f1a P 69870ff4ad6a44ecb692fb40428dec06: Neither blocks nor log segments found. Creating new log.
I20260619 10:20:14.239626 17147 log.cc:824] T ff30921c89da49ae958cbe5701678f1a P 69870ff4ad6a44ecb692fb40428dec06: Log is configured to *not* fsync() on all Append() calls
I20260619 10:20:14.240393 17147 tablet_bootstrap.cc:492] T ff30921c89da49ae958cbe5701678f1a P 69870ff4ad6a44ecb692fb40428dec06: No bootstrap required, opened a new log
I20260619 10:20:14.240458 17147 ts_tablet_manager.cc:1397] T ff30921c89da49ae958cbe5701678f1a P 69870ff4ad6a44ecb692fb40428dec06: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260619 10:20:14.241708 17147 raft_consensus.cc:348] T ff30921c89da49ae958cbe5701678f1a P 69870ff4ad6a44ecb692fb40428dec06 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69870ff4ad6a44ecb692fb40428dec06" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 34349 } } peers { permanent_uuid: "c1b395dd05a64924a62063d847ac8bb2" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 38085 } } peers { permanent_uuid: "d835bbc06b3d47f7b10e69f31a298ed9" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 41351 } }
I20260619 10:20:14.241852 17147 raft_consensus.cc:374] T ff30921c89da49ae958cbe5701678f1a P 69870ff4ad6a44ecb692fb40428dec06 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 10:20:14.241897 17147 raft_consensus.cc:729] T ff30921c89da49ae958cbe5701678f1a P 69870ff4ad6a44ecb692fb40428dec06 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 69870ff4ad6a44ecb692fb40428dec06, State: Initialized, Role: FOLLOWER
I20260619 10:20:14.242022 17147 consensus_queue.cc:260] T ff30921c89da49ae958cbe5701678f1a P 69870ff4ad6a44ecb692fb40428dec06 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69870ff4ad6a44ecb692fb40428dec06" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 34349 } } peers { permanent_uuid: "c1b395dd05a64924a62063d847ac8bb2" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 38085 } } peers { permanent_uuid: "d835bbc06b3d47f7b10e69f31a298ed9" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 41351 } }
I20260619 10:20:14.242300 17147 ts_tablet_manager.cc:1428] T ff30921c89da49ae958cbe5701678f1a P 69870ff4ad6a44ecb692fb40428dec06: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260619 10:20:14.242296 17003 heartbeater.cc:503] Master 127.0.206.126:33511 was elected leader, sending a full tablet report...
W20260619 10:20:14.323550 17004 tablet.cc:2369] T ff30921c89da49ae958cbe5701678f1a P 69870ff4ad6a44ecb692fb40428dec06: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260619 10:20:14.430009 16866 tablet.cc:2369] T ff30921c89da49ae958cbe5701678f1a P d835bbc06b3d47f7b10e69f31a298ed9: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 10:20:14.437915 16871 raft_consensus.cc:482] T ff30921c89da49ae958cbe5701678f1a P d835bbc06b3d47f7b10e69f31a298ed9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 10:20:14.438009 16871 raft_consensus.cc:504] T ff30921c89da49ae958cbe5701678f1a P d835bbc06b3d47f7b10e69f31a298ed9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69870ff4ad6a44ecb692fb40428dec06" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 34349 } } peers { permanent_uuid: "c1b395dd05a64924a62063d847ac8bb2" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 38085 } } peers { permanent_uuid: "d835bbc06b3d47f7b10e69f31a298ed9" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 41351 } }
I20260619 10:20:14.438313 16871 leader_election.cc:288] T ff30921c89da49ae958cbe5701678f1a P d835bbc06b3d47f7b10e69f31a298ed9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 69870ff4ad6a44ecb692fb40428dec06 (127.0.206.66:34349), c1b395dd05a64924a62063d847ac8bb2 (127.0.206.67:38085)
I20260619 10:20:14.441237 17142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:20:14.438667 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.65:54059 (local address 127.0.206.67:38085)
0619 10:20:14.438719 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0619 10:20:14.438721 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 10:20:14.438730 (+ 9us) server_negotiation.cc:407] Connection header received
0619 10:20:14.438810 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:20:14.438811 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:20:14.438818 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:20:14.438885 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 10:20:14.439032 (+ 147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:20:14.439443 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:20:14.440515 (+ 1072us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:20:14.440870 (+ 355us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:20:14.440938 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:20:14.440944 (+ 6us) server_negotiation.cc:1035] Waiting for connection context
0619 10:20:14.441120 (+ 176us) server_negotiation.cc:299] Negotiation successful
0619 10:20:14.441156 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260619 10:20:14.441239 17152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:20:14.438530 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.67:38085 (local address 127.0.206.65:54059)
0619 10:20:14.438669 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0619 10:20:14.438680 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 10:20:14.438734 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 10:20:14.438891 (+ 157us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 10:20:14.438892 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 10:20:14.438894 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 10:20:14.438956 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:20:14.438958 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:20:14.439510 (+ 552us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:20:14.439512 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:20:14.440399 (+ 887us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:20:14.440401 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:20:14.440959 (+ 558us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:20:14.440961 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:20:14.441029 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:20:14.441053 (+ 24us) client_negotiation.cc:769] Sending connection context
0619 10:20:14.441111 (+ 58us) client_negotiation.cc:240] Negotiation successful
0619 10:20:14.441142 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":53,"threads_started":1}
I20260619 10:20:14.441352 17151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:20:14.438476 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.66:34349 (local address 127.0.206.65:34765)
0619 10:20:14.438605 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0619 10:20:14.438632 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0619 10:20:14.438705 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 10:20:14.438880 (+ 175us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 10:20:14.438881 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 10:20:14.438883 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 10:20:14.438940 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:20:14.438946 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:20:14.439589 (+ 643us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:20:14.439590 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:20:14.440401 (+ 811us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:20:14.440403 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:20:14.441102 (+ 699us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:20:14.441103 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:20:14.441160 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:20:14.441163 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 10:20:14.441220 (+ 57us) client_negotiation.cc:240] Negotiation successful
0619 10:20:14.441263 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113,"mutex_wait_us":5,"thread_start_us":66,"threads_started":1}
I20260619 10:20:14.441372 17141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:20:14.438548 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.65:34765 (local address 127.0.206.66:34349)
0619 10:20:14.438664 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0619 10:20:14.438666 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 10:20:14.438680 (+ 14us) server_negotiation.cc:407] Connection header received
0619 10:20:14.438801 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:20:14.438803 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:20:14.438812 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:20:14.438864 (+ 52us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 10:20:14.439004 (+ 140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:20:14.439515 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:20:14.440499 (+ 984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:20:14.440991 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:20:14.441059 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:20:14.441067 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0619 10:20:14.441241 (+ 174us) server_negotiation.cc:299] Negotiation successful
0619 10:20:14.441284 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260619 10:20:14.441567 17089 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ff30921c89da49ae958cbe5701678f1a" candidate_uuid: "d835bbc06b3d47f7b10e69f31a298ed9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c1b395dd05a64924a62063d847ac8bb2" is_pre_election: true
I20260619 10:20:14.441735 17089 raft_consensus.cc:2454] T ff30921c89da49ae958cbe5701678f1a P c1b395dd05a64924a62063d847ac8bb2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d835bbc06b3d47f7b10e69f31a298ed9 in term 0.
I20260619 10:20:14.441761 16957 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ff30921c89da49ae958cbe5701678f1a" candidate_uuid: "d835bbc06b3d47f7b10e69f31a298ed9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "69870ff4ad6a44ecb692fb40428dec06" is_pre_election: true
I20260619 10:20:14.441887 16957 raft_consensus.cc:2454] T ff30921c89da49ae958cbe5701678f1a P 69870ff4ad6a44ecb692fb40428dec06 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d835bbc06b3d47f7b10e69f31a298ed9 in term 0.
I20260619 10:20:14.441968 16754 leader_election.cc:302] T ff30921c89da49ae958cbe5701678f1a P d835bbc06b3d47f7b10e69f31a298ed9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c1b395dd05a64924a62063d847ac8bb2, d835bbc06b3d47f7b10e69f31a298ed9; no voters:
I20260619 10:20:14.442107 16871 raft_consensus.cc:2781] T ff30921c89da49ae958cbe5701678f1a P d835bbc06b3d47f7b10e69f31a298ed9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 10:20:14.442171 16871 raft_consensus.cc:482] T ff30921c89da49ae958cbe5701678f1a P d835bbc06b3d47f7b10e69f31a298ed9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 10:20:14.442193 16871 raft_consensus.cc:3037] T ff30921c89da49ae958cbe5701678f1a P d835bbc06b3d47f7b10e69f31a298ed9 [term 0 FOLLOWER]: Advancing to term 1
W20260619 10:20:14.442514 17136 tablet.cc:2369] T ff30921c89da49ae958cbe5701678f1a P c1b395dd05a64924a62063d847ac8bb2: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 10:20:14.442915 16871 raft_consensus.cc:504] T ff30921c89da49ae958cbe5701678f1a P d835bbc06b3d47f7b10e69f31a298ed9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69870ff4ad6a44ecb692fb40428dec06" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 34349 } } peers { permanent_uuid: "c1b395dd05a64924a62063d847ac8bb2" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 38085 } } peers { permanent_uuid: "d835bbc06b3d47f7b10e69f31a298ed9" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 41351 } }
I20260619 10:20:14.443039 16871 leader_election.cc:288] T ff30921c89da49ae958cbe5701678f1a P d835bbc06b3d47f7b10e69f31a298ed9 [CANDIDATE]: Term 1 election: Requested vote from peers 69870ff4ad6a44ecb692fb40428dec06 (127.0.206.66:34349), c1b395dd05a64924a62063d847ac8bb2 (127.0.206.67:38085)
I20260619 10:20:14.443166 16957 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ff30921c89da49ae958cbe5701678f1a" candidate_uuid: "d835bbc06b3d47f7b10e69f31a298ed9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "69870ff4ad6a44ecb692fb40428dec06"
I20260619 10:20:14.443187 17089 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ff30921c89da49ae958cbe5701678f1a" candidate_uuid: "d835bbc06b3d47f7b10e69f31a298ed9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c1b395dd05a64924a62063d847ac8bb2"
I20260619 10:20:14.443240 16957 raft_consensus.cc:3037] T ff30921c89da49ae958cbe5701678f1a P 69870ff4ad6a44ecb692fb40428dec06 [term 0 FOLLOWER]: Advancing to term 1
I20260619 10:20:14.443245 17089 raft_consensus.cc:3037] T ff30921c89da49ae958cbe5701678f1a P c1b395dd05a64924a62063d847ac8bb2 [term 0 FOLLOWER]: Advancing to term 1
I20260619 10:20:14.444113 17089 raft_consensus.cc:2454] T ff30921c89da49ae958cbe5701678f1a P c1b395dd05a64924a62063d847ac8bb2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d835bbc06b3d47f7b10e69f31a298ed9 in term 1.
I20260619 10:20:14.444139 16957 raft_consensus.cc:2454] T ff30921c89da49ae958cbe5701678f1a P 69870ff4ad6a44ecb692fb40428dec06 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d835bbc06b3d47f7b10e69f31a298ed9 in term 1.
I20260619 10:20:14.444308 16752 leader_election.cc:302] T ff30921c89da49ae958cbe5701678f1a P d835bbc06b3d47f7b10e69f31a298ed9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 69870ff4ad6a44ecb692fb40428dec06, d835bbc06b3d47f7b10e69f31a298ed9; no voters:
I20260619 10:20:14.444393 16871 raft_consensus.cc:2781] T ff30921c89da49ae958cbe5701678f1a P d835bbc06b3d47f7b10e69f31a298ed9 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 10:20:14.444554 16871 raft_consensus.cc:686] T ff30921c89da49ae958cbe5701678f1a P d835bbc06b3d47f7b10e69f31a298ed9 [term 1 LEADER]: Becoming Leader. State: Replica: d835bbc06b3d47f7b10e69f31a298ed9, State: Running, Role: LEADER
I20260619 10:20:14.444626 16871 consensus_queue.cc:237] T ff30921c89da49ae958cbe5701678f1a P d835bbc06b3d47f7b10e69f31a298ed9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69870ff4ad6a44ecb692fb40428dec06" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 34349 } } peers { permanent_uuid: "c1b395dd05a64924a62063d847ac8bb2" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 38085 } } peers { permanent_uuid: "d835bbc06b3d47f7b10e69f31a298ed9" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 41351 } }
I20260619 10:20:14.445461 16677 catalog_manager.cc:5495] T ff30921c89da49ae958cbe5701678f1a P d835bbc06b3d47f7b10e69f31a298ed9 reported cstate change: term changed from 0 to 1, leader changed from <none> to d835bbc06b3d47f7b10e69f31a298ed9 (127.0.206.65). New cstate: current_term: 1 leader_uuid: "d835bbc06b3d47f7b10e69f31a298ed9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69870ff4ad6a44ecb692fb40428dec06" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 34349 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c1b395dd05a64924a62063d847ac8bb2" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 38085 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d835bbc06b3d47f7b10e69f31a298ed9" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 41351 } health_report { overall_health: HEALTHY } } }
I20260619 10:20:14.452781 17156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:20:14.450865 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58420 (local address 127.0.206.65:41351)
0619 10:20:14.450966 (+ 101us) server_negotiation.cc:206] Beginning negotiation
0619 10:20:14.450967 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 10:20:14.450984 (+ 17us) server_negotiation.cc:407] Connection header received
0619 10:20:14.451006 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:20:14.451006 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:20:14.451019 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:20:14.451071 (+ 52us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 10:20:14.451144 (+ 73us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:20:14.451522 (+ 378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:20:14.452214 (+ 692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:20:14.452375 (+ 161us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:20:14.452419 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:20:14.452531 (+ 112us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 10:20:14.452610 (+ 79us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 10:20:14.452655 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0619 10:20:14.452672 (+ 17us) server_negotiation.cc:299] Negotiation successful
0619 10:20:14.452685 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87,"thread_start_us":59,"threads_started":1}
I20260619 10:20:14.454021 16957 raft_consensus.cc:1261] T ff30921c89da49ae958cbe5701678f1a P 69870ff4ad6a44ecb692fb40428dec06 [term 1 FOLLOWER]: Refusing update from remote peer d835bbc06b3d47f7b10e69f31a298ed9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 10:20:14.454020 17089 raft_consensus.cc:1261] T ff30921c89da49ae958cbe5701678f1a P c1b395dd05a64924a62063d847ac8bb2 [term 1 FOLLOWER]: Refusing update from remote peer d835bbc06b3d47f7b10e69f31a298ed9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 10:20:14.454376 17153 consensus_queue.cc:1035] T ff30921c89da49ae958cbe5701678f1a P d835bbc06b3d47f7b10e69f31a298ed9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "69870ff4ad6a44ecb692fb40428dec06" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 34349 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 10:20:14.454509 16871 consensus_queue.cc:1035] T ff30921c89da49ae958cbe5701678f1a P d835bbc06b3d47f7b10e69f31a298ed9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1b395dd05a64924a62063d847ac8bb2" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 38085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 10:20:14.457605 17159 mvcc.cc:205] Tried to move back new op lower bound from 7298516641601368064 to 7298516641565855744. Current Snapshot: MvccSnapshot[applied={T|T < 7298516641601368064 or (T in {7298516641601368064})}]
I20260619 10:20:14.458837 16676 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47886:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 10:20:14.458930 16676 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47886:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 10:20:14.459589 16676 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b4330665f37e4f6693f0900bed7dcfe6: Sending DeleteTablet for 3 replicas of tablet ff30921c89da49ae958cbe5701678f1a
I20260619 10:20:14.459920 16937 tablet_service.cc:1514] Processing DeleteTablet for tablet ff30921c89da49ae958cbe5701678f1a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 10:20:14 UTC) from {username='kudu', principal='kudu/127.0.206.126@KRBTEST.COM'} at 127.0.0.1:46994
I20260619 10:20:14.459975 17069 tablet_service.cc:1514] Processing DeleteTablet for tablet ff30921c89da49ae958cbe5701678f1a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 10:20:14 UTC) from {username='kudu', principal='kudu/127.0.206.126@KRBTEST.COM'} at 127.0.0.1:48550
I20260619 10:20:14.460007 825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTU9OP6/build/release/bin/kudu with pid 16715
I20260619 10:20:14.460155 17168 tablet_replica.cc:330] T ff30921c89da49ae958cbe5701678f1a P c1b395dd05a64924a62063d847ac8bb2: stopping tablet replica
I20260619 10:20:14.460259 17167 tablet_replica.cc:330] T ff30921c89da49ae958cbe5701678f1a P 69870ff4ad6a44ecb692fb40428dec06: stopping tablet replica
I20260619 10:20:14.460269 17168 raft_consensus.cc:2229] T ff30921c89da49ae958cbe5701678f1a P c1b395dd05a64924a62063d847ac8bb2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 10:20:14.460369 17168 raft_consensus.cc:2258] T ff30921c89da49ae958cbe5701678f1a P c1b395dd05a64924a62063d847ac8bb2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 10:20:14.460453 17167 raft_consensus.cc:2229] T ff30921c89da49ae958cbe5701678f1a P 69870ff4ad6a44ecb692fb40428dec06 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 10:20:14.460660 17167 raft_consensus.cc:2258] T ff30921c89da49ae958cbe5701678f1a P 69870ff4ad6a44ecb692fb40428dec06 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 10:20:14.460960 17168 ts_tablet_manager.cc:1905] T ff30921c89da49ae958cbe5701678f1a P c1b395dd05a64924a62063d847ac8bb2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 10:20:14.461066 17167 ts_tablet_manager.cc:1905] T ff30921c89da49ae958cbe5701678f1a P 69870ff4ad6a44ecb692fb40428dec06: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 10:20:14.462585 17168 ts_tablet_manager.cc:1918] T ff30921c89da49ae958cbe5701678f1a P c1b395dd05a64924a62063d847ac8bb2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 10:20:14.462585 17167 ts_tablet_manager.cc:1918] T ff30921c89da49ae958cbe5701678f1a P 69870ff4ad6a44ecb692fb40428dec06: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 10:20:14.462673 17167 log.cc:1196] T ff30921c89da49ae958cbe5701678f1a P 69870ff4ad6a44ecb692fb40428dec06: Deleting WAL directory at /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781864224577058-825-0/minicluster-data/ts-1/wal/wals/ff30921c89da49ae958cbe5701678f1a
I20260619 10:20:14.462673 17168 log.cc:1196] T ff30921c89da49ae958cbe5701678f1a P c1b395dd05a64924a62063d847ac8bb2: Deleting WAL directory at /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781864224577058-825-0/minicluster-data/ts-2/wal/wals/ff30921c89da49ae958cbe5701678f1a
I20260619 10:20:14.462965 17167 ts_tablet_manager.cc:1939] T ff30921c89da49ae958cbe5701678f1a P 69870ff4ad6a44ecb692fb40428dec06: Deleting consensus metadata
I20260619 10:20:14.463038 17168 ts_tablet_manager.cc:1939] T ff30921c89da49ae958cbe5701678f1a P c1b395dd05a64924a62063d847ac8bb2: Deleting consensus metadata
I20260619 10:20:14.463352 16662 catalog_manager.cc:4841] TS 69870ff4ad6a44ecb692fb40428dec06 (127.0.206.66:34349): tablet ff30921c89da49ae958cbe5701678f1a (table test-table [id=ab9a01d8ddb943bb8e5d5246dd265fd8]) successfully deleted
I20260619 10:20:14.463395 16664 catalog_manager.cc:4841] TS c1b395dd05a64924a62063d847ac8bb2 (127.0.206.67:38085): tablet ff30921c89da49ae958cbe5701678f1a (table test-table [id=ab9a01d8ddb943bb8e5d5246dd265fd8]) successfully deleted
W20260619 10:20:14.465919 16661 connection.cc:535] client connection to 127.0.206.65:41351 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260619 10:20:14.466038 16661 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260619 10:20:14.466495 825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTU9OP6/build/release/bin/kudu with pid 16870
I20260619 10:20:14.466511 17143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:20:14.466400 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.65:41351 (local address 127.0.0.1:58422)
0619 10:20:14.466453 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0619 10:20:14.466486 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.65:41351: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260619 10:20:14.467737 16661 catalog_manager.cc:4584] TS d835bbc06b3d47f7b10e69f31a298ed9 (127.0.206.65:41351): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet ff30921c89da49ae958cbe5701678f1a: Network error: Client connection negotiation failed: client connection to 127.0.206.65:41351: connect: Connection refused (error 111)
I20260619 10:20:14.471479 825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTU9OP6/build/release/bin/kudu with pid 17006
I20260619 10:20:14.475627 825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTU9OP6/build/release/bin/kudu with pid 16644
I20260619 10:20:34.480736 825 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTU9OP6/build/release/bin/kudu
/tmp/dist-test-taskTU9OP6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781864224577058-825-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781864224577058-825-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781864224577058-825-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781864224577058-825-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.206.126:33511
--webserver_interface=127.0.206.126
--webserver_port=43223
--keytab_file=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781864224577058-825-0/krb5kdc/kudu_127.0.206.126.keytab
--principal=kudu/127.0.206.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.206.84:43867
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.206.126:33511
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781864224577058-825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781864224577058-825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781864224577058-825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 10:20:34.559265 17170 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 10:20:34.559465 17170 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 10:20:34.559486 17170 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 10:20:34.560611 17170 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260619 10:20:34.560649 17170 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260619 10:20:34.560662 17170 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260619 10:20:34.560701 17170 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 10:20:34.560729 17170 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 10:20:34.560753 17170 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 10:20:34.560779 17170 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 10:20:34.560808 17170 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 10:20:34.562263 17170 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.206.84:43867
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781864224577058-825-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781864224577058-825-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.0.206.126:33511
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781864224577058-825-0/krb5kdc/kudu_127.0.206.126.keytab
--principal=kudu/127.0.206.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.206.126:33511
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781864224577058-825-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.206.126
--webserver_port=43223
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17170
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781864224577058-825-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 09:32:44 UTC on ccycloud.quasar-gbtldp.root.comops.site
build id 4455440
I20260619 10:20:34.562508 17170 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 10:20:34.562794 17170 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 10:20:34.567307 17178 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 10:20:34.567313 17175 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 10:20:34.567662 17170 server_base.cc:1034] running on GCE node
W20260619 10:20:34.567313 17176 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 10:20:34.567962 17170 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 10:20:34.568231 17170 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 10:20:34.569394 17170 hybrid_clock.cc:648] HybridClock initialized: now 1781864434569386 us; error 32 us; skew 500 ppm
Jun 19 10:20:34 dist-test-slave-6pt2 krb5kdc[16099](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864434, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 10:20:34.576299 17170 init.cc:377] Logged in from keytab as kudu/127.0.206.126@KRBTEST.COM (short username kudu)
I20260619 10:20:34.576695 17170 webserver.cc:460] Webserver started at http://127.0.206.126:43223/ using document root <none> and password file <none>
I20260619 10:20:34.576921 17170 fs_manager.cc:359] Metadata directory not provided
I20260619 10:20:34.576985 17170 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 10:20:34.578253 17170 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 10:20:34.578922 17185 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 10:20:34.579070 17170 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260619 10:20:34.579164 17170 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781864224577058-825-0/minicluster-data/master-0/data,/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781864224577058-825-0/minicluster-data/master-0/wal
uuid: "b4330665f37e4f6693f0900bed7dcfe6"
format_stamp: "Formatted at 2026-06-19 10:19:52 on dist-test-slave-6pt2"
server_key: "1b36e4882d0518269926abeae9ba3acc"
server_key_iv: "8dd3ce24ed149b1d6aab6338fcf9b6df"
server_key_version: "encryptionkey@0"
I20260619 10:20:34.579469 17170 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781864224577058-825-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781864224577058-825-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781864224577058-825-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 10:20:34.598496 17170 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 10:20:34.600757 17170 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 10:20:34.600917 17170 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 10:20:34.604732 17170 rpc_server.cc:307] RPC server started. Bound to: 127.0.206.126:33511
I20260619 10:20:34.604784 17237 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.206.126:33511 every 8 connection(s)
I20260619 10:20:34.605250 17170 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781864224577058-825-0/minicluster-data/master-0/data/info.pb
I20260619 10:20:34.605743 825 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTU9OP6/build/release/bin/kudu as pid 17170
I20260619 10:20:34.606106 825 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTU9OP6/build/release/bin/kudu
/tmp/dist-test-taskTU9OP6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781864224577058-825-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781864224577058-825-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781864224577058-825-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781864224577058-825-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.206.65:41351
--local_ip_for_outbound_sockets=127.0.206.65
--tserver_master_addrs=127.0.206.126:33511
--webserver_port=38183
--webserver_interface=127.0.206.65
--keytab_file=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781864224577058-825-0/krb5kdc/kudu_127.0.206.65.keytab
--principal=kudu/127.0.206.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.206.84:43867
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781864224577058-825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781864224577058-825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781864224577058-825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 10:20:34.607493 17238 sys_catalog.cc:263] Verifying existing consensus state
I20260619 10:20:34.608497 17238 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b4330665f37e4f6693f0900bed7dcfe6: Bootstrap starting.
I20260619 10:20:34.618157 17238 log.cc:824] T 00000000000000000000000000000000 P b4330665f37e4f6693f0900bed7dcfe6: Log is configured to *not* fsync() on all Append() calls
I20260619 10:20:34.620263 17238 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b4330665f37e4f6693f0900bed7dcfe6: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260619 10:20:34.620556 17238 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b4330665f37e4f6693f0900bed7dcfe6: Bootstrap complete.
I20260619 10:20:34.622836 17238 raft_consensus.cc:348] T 00000000000000000000000000000000 P b4330665f37e4f6693f0900bed7dcfe6 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b4330665f37e4f6693f0900bed7dcfe6" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 33511 } }
I20260619 10:20:34.623044 17238 raft_consensus.cc:729] T 00000000000000000000000000000000 P b4330665f37e4f6693f0900bed7dcfe6 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: b4330665f37e4f6693f0900bed7dcfe6, State: Initialized, Role: FOLLOWER
I20260619 10:20:34.623198 17238 consensus_queue.cc:260] T 00000000000000000000000000000000 P b4330665f37e4f6693f0900bed7dcfe6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b4330665f37e4f6693f0900bed7dcfe6" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 33511 } }
I20260619 10:20:34.623251 17238 raft_consensus.cc:388] T 00000000000000000000000000000000 P b4330665f37e4f6693f0900bed7dcfe6 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 10:20:34.623308 17238 raft_consensus.cc:482] T 00000000000000000000000000000000 P b4330665f37e4f6693f0900bed7dcfe6 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 10:20:34.623359 17238 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b4330665f37e4f6693f0900bed7dcfe6 [term 2 FOLLOWER]: Advancing to term 3
I20260619 10:20:34.624258 17238 raft_consensus.cc:504] T 00000000000000000000000000000000 P b4330665f37e4f6693f0900bed7dcfe6 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b4330665f37e4f6693f0900bed7dcfe6" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 33511 } }
I20260619 10:20:34.624368 17238 leader_election.cc:302] T 00000000000000000000000000000000 P b4330665f37e4f6693f0900bed7dcfe6 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b4330665f37e4f6693f0900bed7dcfe6; no voters:
I20260619 10:20:34.624547 17238 leader_election.cc:288] T 00000000000000000000000000000000 P b4330665f37e4f6693f0900bed7dcfe6 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260619 10:20:34.624577 17242 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b4330665f37e4f6693f0900bed7dcfe6 [term 3 FOLLOWER]: Leader election won for term 3
I20260619 10:20:34.624696 17242 raft_consensus.cc:686] T 00000000000000000000000000000000 P b4330665f37e4f6693f0900bed7dcfe6 [term 3 LEADER]: Becoming Leader. State: Replica: b4330665f37e4f6693f0900bed7dcfe6, State: Running, Role: LEADER
I20260619 10:20:34.624790 17238 sys_catalog.cc:564] T 00000000000000000000000000000000 P b4330665f37e4f6693f0900bed7dcfe6 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 10:20:34.624774 17242 consensus_queue.cc:237] T 00000000000000000000000000000000 P b4330665f37e4f6693f0900bed7dcfe6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b4330665f37e4f6693f0900bed7dcfe6" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 33511 } }
I20260619 10:20:34.625061 17242 sys_catalog.cc:455] T 00000000000000000000000000000000 P b4330665f37e4f6693f0900bed7dcfe6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b4330665f37e4f6693f0900bed7dcfe6. Latest consensus state: current_term: 3 leader_uuid: "b4330665f37e4f6693f0900bed7dcfe6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b4330665f37e4f6693f0900bed7dcfe6" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 33511 } } }
I20260619 10:20:34.625137 17242 sys_catalog.cc:458] T 00000000000000000000000000000000 P b4330665f37e4f6693f0900bed7dcfe6 [sys.catalog]: This master's current role is: LEADER
I20260619 10:20:34.625064 17244 sys_catalog.cc:455] T 00000000000000000000000000000000 P b4330665f37e4f6693f0900bed7dcfe6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "b4330665f37e4f6693f0900bed7dcfe6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b4330665f37e4f6693f0900bed7dcfe6" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 33511 } } }
I20260619 10:20:34.625286 17244 sys_catalog.cc:458] T 00000000000000000000000000000000 P b4330665f37e4f6693f0900bed7dcfe6 [sys.catalog]: This master's current role is: LEADER
I20260619 10:20:34.626706 17258 catalog_manager.cc:1259] Loaded cluster ID: a2afe9084c3b435aaa7c67b9feec0afd
I20260619 10:20:34.626741 17258 catalog_manager.cc:1552] T 00000000000000000000000000000000 P b4330665f37e4f6693f0900bed7dcfe6: loading cluster ID for follower catalog manager: success
I20260619 10:20:34.627491 17258 catalog_manager.cc:1574] T 00000000000000000000000000000000 P b4330665f37e4f6693f0900bed7dcfe6: acquiring CA information for follower catalog manager: success
F20260619 10:20:34.627815 17258 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f102ea979ad google::LogMessage::Fail() at ??:0
@ 0x7f102ea9a88a google::LogMessage::SendToLog() at ??:0
@ 0x7f102ea974de google::LogMessage::Flush() at ??:0
@ 0x7f102ea98a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f1031204104 kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7f10369f06b9 kudu::master::CatalogManager::LoadTspkEntries() at ??:0
@ 0x7f10369f0989 kudu::master::CatalogManager::PrepareFollowerTokenVerifier() at ??:0
@ 0x7f10369f0e4e kudu::master::CatalogManager::PrepareFollower() at ??:0
@ 0x7f10369f38c2 kudu::master::CatalogManagerBgTasks::Run() at ??:0
@ 0x7f103095a954 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f10317be6db start_thread at ??:0
@ 0x7f102d59561f clone at ??:0
W20260619 10:20:34.703907 17240 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 10:20:34.704097 17240 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 10:20:34.704121 17240 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 10:20:34.705263 17240 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 10:20:34.705309 17240 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 10:20:34.705336 17240 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.206.65
I20260619 10:20:34.706792 17240 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.206.84:43867
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781864224577058-825-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781864224577058-825-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781864224577058-825-0/krb5kdc/kudu_127.0.206.65.keytab
--principal=kudu/127.0.206.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.206.65:41351
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781864224577058-825-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.206.65
--webserver_port=38183
--tserver_master_addrs=127.0.206.126:33511
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17240
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.206.65
--log_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781864224577058-825-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 09:32:44 UTC on ccycloud.quasar-gbtldp.root.comops.site
build id 4455440
I20260619 10:20:34.707015 17240 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 10:20:34.707255 17240 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 10:20:34.712020 17276 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 10:20:34.712255 17279 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 10:20:34.712339 17240 server_base.cc:1034] running on GCE node
W20260619 10:20:34.712347 17277 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 10:20:34.712615 17240 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 10:20:34.712853 17240 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 10:20:34.714017 17240 hybrid_clock.cc:648] HybridClock initialized: now 1781864434714005 us; error 31 us; skew 500 ppm
Jun 19 10:20:34 dist-test-slave-6pt2 krb5kdc[16099](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864434, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 10:20:34.716576 17240 init.cc:377] Logged in from keytab as kudu/127.0.206.65@KRBTEST.COM (short username kudu)
I20260619 10:20:34.717011 17240 webserver.cc:460] Webserver started at http://127.0.206.65:38183/ using document root <none> and password file <none>
I20260619 10:20:34.717249 17240 fs_manager.cc:359] Metadata directory not provided
I20260619 10:20:34.717319 17240 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 10:20:34.718713 17240 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260619 10:20:34.719429 17286 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 10:20:34.719681 17240 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260619 10:20:34.719784 17240 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781864224577058-825-0/minicluster-data/ts-0/data,/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781864224577058-825-0/minicluster-data/ts-0/wal
uuid: "d835bbc06b3d47f7b10e69f31a298ed9"
format_stamp: "Formatted at 2026-06-19 10:19:53 on dist-test-slave-6pt2"
server_key: "96107af8396e60298dc6ab4457b9e42a"
server_key_iv: "8ffac846010f0255650acda1a7ead878"
server_key_version: "encryptionkey@0"
I20260619 10:20:34.720114 17240 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781864224577058-825-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781864224577058-825-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781864224577058-825-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 10:20:34.730948 17240 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 10:20:34.733135 17240 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 10:20:34.733284 17240 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 10:20:34.733501 17240 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 10:20:34.733928 17293 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260619 10:20:34.736510 17240 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260619 10:20:34.736562 17240 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.003s user 0.000s sys 0.000s
I20260619 10:20:34.736603 17240 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260619 10:20:34.737181 17240 ts_tablet_manager.cc:610] Registered 1 tablets
I20260619 10:20:34.737262 17240 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260619 10:20:34.737315 17293 tablet_bootstrap.cc:492] T ff30921c89da49ae958cbe5701678f1a P d835bbc06b3d47f7b10e69f31a298ed9: Bootstrap starting.
I20260619 10:20:34.744294 17240 rpc_server.cc:307] RPC server started. Bound to: 127.0.206.65:41351
I20260619 10:20:34.744437 17400 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.206.65:41351 every 8 connection(s)
I20260619 10:20:34.744678 17240 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781864224577058-825-0/minicluster-data/ts-0/data/info.pb
I20260619 10:20:34.748865 17293 log.cc:824] T ff30921c89da49ae958cbe5701678f1a P d835bbc06b3d47f7b10e69f31a298ed9: Log is configured to *not* fsync() on all Append() calls
I20260619 10:20:34.751768 17293 tablet_bootstrap.cc:492] T ff30921c89da49ae958cbe5701678f1a P d835bbc06b3d47f7b10e69f31a298ed9: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260619 10:20:34.752200 825 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTU9OP6/build/release/bin/kudu as pid 17240
I20260619 10:20:34.752199 17293 tablet_bootstrap.cc:492] T ff30921c89da49ae958cbe5701678f1a P d835bbc06b3d47f7b10e69f31a298ed9: Bootstrap complete.
I20260619 10:20:34.752483 825 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTU9OP6/build/release/bin/kudu
/tmp/dist-test-taskTU9OP6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781864224577058-825-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781864224577058-825-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781864224577058-825-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781864224577058-825-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.206.66:34349
--local_ip_for_outbound_sockets=127.0.206.66
--tserver_master_addrs=127.0.206.126:33511
--webserver_port=39951
--webserver_interface=127.0.206.66
--keytab_file=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781864224577058-825-0/krb5kdc/kudu_127.0.206.66.keytab
--principal=kudu/127.0.206.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.206.84:43867
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781864224577058-825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781864224577058-825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781864224577058-825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 10:20:34.753440 17293 ts_tablet_manager.cc:1397] T ff30921c89da49ae958cbe5701678f1a P d835bbc06b3d47f7b10e69f31a298ed9: Time spent bootstrapping tablet: real 0.016s user 0.001s sys 0.012s
I20260619 10:20:34.755064 17293 raft_consensus.cc:348] T ff30921c89da49ae958cbe5701678f1a P d835bbc06b3d47f7b10e69f31a298ed9 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69870ff4ad6a44ecb692fb40428dec06" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 34349 } } peers { permanent_uuid: "c1b395dd05a64924a62063d847ac8bb2" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 38085 } } peers { permanent_uuid: "d835bbc06b3d47f7b10e69f31a298ed9" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 41351 } }
I20260619 10:20:34.755641 17293 raft_consensus.cc:729] T ff30921c89da49ae958cbe5701678f1a P d835bbc06b3d47f7b10e69f31a298ed9 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: d835bbc06b3d47f7b10e69f31a298ed9, State: Initialized, Role: FOLLOWER
I20260619 10:20:34.755844 17293 consensus_queue.cc:260] T ff30921c89da49ae958cbe5701678f1a P d835bbc06b3d47f7b10e69f31a298ed9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69870ff4ad6a44ecb692fb40428dec06" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 34349 } } peers { permanent_uuid: "c1b395dd05a64924a62063d847ac8bb2" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 38085 } } peers { permanent_uuid: "d835bbc06b3d47f7b10e69f31a298ed9" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 41351 } }
I20260619 10:20:34.756151 17293 ts_tablet_manager.cc:1428] T ff30921c89da49ae958cbe5701678f1a P d835bbc06b3d47f7b10e69f31a298ed9: Time spent starting tablet: real 0.003s user 0.005s sys 0.000s
W20260619 10:20:34.830245 17405 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 10:20:34.830468 17405 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 10:20:34.830505 17405 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 10:20:34.831693 17405 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 10:20:34.831755 17405 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 10:20:34.831795 17405 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.206.66
I20260619 10:20:34.834715 17405 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.206.84:43867
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781864224577058-825-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781864224577058-825-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781864224577058-825-0/krb5kdc/kudu_127.0.206.66.keytab
--principal=kudu/127.0.206.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.206.66:34349
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781864224577058-825-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.206.66
--webserver_port=39951
--tserver_master_addrs=127.0.206.126:33511
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17405
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.206.66
--log_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781864224577058-825-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 09:32:44 UTC on ccycloud.quasar-gbtldp.root.comops.site
build id 4455440
I20260619 10:20:34.834954 17405 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 10:20:34.835192 17405 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 10:20:34.836959 17403 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 10:20:34.745293 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:33511 (local address 127.0.206.65:53083)
0619 10:20:34.745437 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0619 10:20:34.745477 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0619 10:20:34.745959 (+ 482us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 10:20:34.836767 (+ 90808us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.126:33511: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":86,"threads_started":1}
W20260619 10:20:34.837239 17401 heartbeater.cc:644] Failed to heartbeat to 127.0.206.126:33511 (0 consecutive failures): Network error: Failed to ping master at 127.0.206.126:33511: Client connection negotiation failed: client connection to 127.0.206.126:33511: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260619 10:20:34.837587 17403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:20:34.837474 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:33511 (local address 127.0.206.65:53513)
0619 10:20:34.837525 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0619 10:20:34.837566 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.126:33511: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
I20260619 10:20:34.837874 17403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:20:34.837801 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:33511 (local address 127.0.206.65:60825)
0619 10:20:34.837837 (+ 36us) negotiation.cc:107] Waiting for socket to connect
0619 10:20:34.837859 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.126:33511: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
W20260619 10:20:34.837977 17401 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260619 10:20:34.840653 17411 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 10:20:34.840618 17412 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 10:20:34.840622 17414 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 10:20:34.840886 17405 server_base.cc:1034] running on GCE node
I20260619 10:20:34.841046 17405 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 10:20:34.841213 17405 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 10:20:34.842334 17405 hybrid_clock.cc:648] HybridClock initialized: now 1781864434842321 us; error 24 us; skew 500 ppm
Jun 19 10:20:34 dist-test-slave-6pt2 krb5kdc[16099](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864434, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 10:20:34.844686 17405 init.cc:377] Logged in from keytab as kudu/127.0.206.66@KRBTEST.COM (short username kudu)
I20260619 10:20:34.845127 17405 webserver.cc:460] Webserver started at http://127.0.206.66:39951/ using document root <none> and password file <none>
I20260619 10:20:34.845342 17405 fs_manager.cc:359] Metadata directory not provided
I20260619 10:20:34.845386 17405 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 10:20:34.846873 17405 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 10:20:34.847433 17421 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 10:20:34.847600 17405 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 10:20:34.847678 17405 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781864224577058-825-0/minicluster-data/ts-1/data,/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781864224577058-825-0/minicluster-data/ts-1/wal
uuid: "69870ff4ad6a44ecb692fb40428dec06"
format_stamp: "Formatted at 2026-06-19 10:19:53 on dist-test-slave-6pt2"
server_key: "47ac82c46eba2f3daf1ebc6486ee0b63"
server_key_iv: "76cc672c9e34ed3da1f5b39f9c85b5fd"
server_key_version: "encryptionkey@0"
I20260619 10:20:34.847983 17405 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781864224577058-825-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781864224577058-825-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781864224577058-825-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 10:20:34.862278 17405 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 10:20:34.864348 17405 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 10:20:34.864487 17405 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 10:20:34.864677 17405 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 10:20:34.864969 17405 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 10:20:34.865005 17405 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 10:20:34.865041 17405 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 10:20:34.865067 17405 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 10:20:34.870745 17405 rpc_server.cc:307] RPC server started. Bound to: 127.0.206.66:34349
I20260619 10:20:34.870786 17534 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.206.66:34349 every 8 connection(s)
I20260619 10:20:34.871081 17405 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781864224577058-825-0/minicluster-data/ts-1/data/info.pb
I20260619 10:20:34.872037 17537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:20:34.871647 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:33511 (local address 127.0.206.66:57947)
0619 10:20:34.871794 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0619 10:20:34.871887 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.126:33511: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":79,"threads_started":1}
W20260619 10:20:34.872421 17535 heartbeater.cc:644] Failed to heartbeat to 127.0.206.126:33511 (0 consecutive failures): Network error: Failed to ping master at 127.0.206.126:33511: Client connection negotiation failed: client connection to 127.0.206.126:33511: connect: Connection refused (error 111)
I20260619 10:20:34.872691 17537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:20:34.872606 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:33511 (local address 127.0.206.66:47325)
0619 10:20:34.872648 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0619 10:20:34.872671 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.126:33511: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
I20260619 10:20:34.872993 17537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:20:34.872919 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:33511 (local address 127.0.206.66:60955)
0619 10:20:34.872957 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0619 10:20:34.872977 (+ 20us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.126:33511: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
W20260619 10:20:34.873118 17535 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260619 10:20:34.877616 825 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTU9OP6/build/release/bin/kudu as pid 17405
I20260619 10:20:34.877892 825 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTU9OP6/build/release/bin/kudu
/tmp/dist-test-taskTU9OP6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781864224577058-825-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781864224577058-825-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781864224577058-825-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781864224577058-825-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.206.67:38085
--local_ip_for_outbound_sockets=127.0.206.67
--tserver_master_addrs=127.0.206.126:33511
--webserver_port=33907
--webserver_interface=127.0.206.67
--keytab_file=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781864224577058-825-0/krb5kdc/kudu_127.0.206.67.keytab
--principal=kudu/127.0.206.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.206.84:43867
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781864224577058-825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781864224577058-825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781864224577058-825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 10:20:34.956513 17538 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 10:20:34.956700 17538 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 10:20:34.956722 17538 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 10:20:34.958060 17538 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 10:20:34.958105 17538 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 10:20:34.958154 17538 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.206.67
I20260619 10:20:34.959581 17538 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.206.84:43867
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781864224577058-825-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781864224577058-825-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781864224577058-825-0/krb5kdc/kudu_127.0.206.67.keytab
--principal=kudu/127.0.206.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.206.67:38085
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781864224577058-825-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.206.67
--webserver_port=33907
--tserver_master_addrs=127.0.206.126:33511
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17538
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.206.67
--log_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781864224577058-825-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 09:32:44 UTC on ccycloud.quasar-gbtldp.root.comops.site
build id 4455440
I20260619 10:20:34.959808 17538 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 10:20:34.960041 17538 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 10:20:34.964848 17543 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 10:20:34.964835 17544 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 10:20:34.964836 17546 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 10:20:34.965003 17538 server_base.cc:1034] running on GCE node
I20260619 10:20:34.965209 17538 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 10:20:34.965446 17538 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 10:20:34.966589 17538 hybrid_clock.cc:648] HybridClock initialized: now 1781864434966569 us; error 34 us; skew 500 ppm
Jun 19 10:20:34 dist-test-slave-6pt2 krb5kdc[16099](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864434, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 10:20:34.968962 17538 init.cc:377] Logged in from keytab as kudu/127.0.206.67@KRBTEST.COM (short username kudu)
I20260619 10:20:34.969429 17538 webserver.cc:460] Webserver started at http://127.0.206.67:33907/ using document root <none> and password file <none>
I20260619 10:20:34.969638 17538 fs_manager.cc:359] Metadata directory not provided
I20260619 10:20:34.969683 17538 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 10:20:34.971055 17538 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 10:20:34.971613 17553 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 10:20:34.971764 17538 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 10:20:34.971858 17538 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781864224577058-825-0/minicluster-data/ts-2/data,/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781864224577058-825-0/minicluster-data/ts-2/wal
uuid: "c1b395dd05a64924a62063d847ac8bb2"
format_stamp: "Formatted at 2026-06-19 10:19:53 on dist-test-slave-6pt2"
server_key: "b86270f85a9aa898f56881c27592156e"
server_key_iv: "e0fdc184a4f5eb447f166ff615c876b4"
server_key_version: "encryptionkey@0"
I20260619 10:20:34.972170 17538 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781864224577058-825-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781864224577058-825-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781864224577058-825-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 10:20:34.988559 17538 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 10:20:34.990166 17538 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 10:20:34.990330 17538 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 10:20:34.990542 17538 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 10:20:34.990849 17538 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 10:20:34.990898 17538 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 10:20:34.990931 17538 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 10:20:34.990957 17538 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 10:20:34.996603 17538 rpc_server.cc:307] RPC server started. Bound to: 127.0.206.67:38085
I20260619 10:20:34.996650 17666 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.206.67:38085 every 8 connection(s)
I20260619 10:20:34.996948 17538 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781864224577058-825-0/minicluster-data/ts-2/data/info.pb
I20260619 10:20:34.997892 17669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:20:34.997557 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:33511 (local address 127.0.206.67:41767)
0619 10:20:34.997696 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0619 10:20:34.997758 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.126:33511: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":70,"threads_started":1}
W20260619 10:20:34.998263 17667 heartbeater.cc:644] Failed to heartbeat to 127.0.206.126:33511 (0 consecutive failures): Network error: Failed to ping master at 127.0.206.126:33511: Client connection negotiation failed: client connection to 127.0.206.126:33511: connect: Connection refused (error 111)
I20260619 10:20:34.998517 17669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:20:34.998438 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:33511 (local address 127.0.206.67:43225)
0619 10:20:34.998477 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0619 10:20:34.998501 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.126:33511: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
I20260619 10:20:34.998754 17669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:20:34.998707 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:33511 (local address 127.0.206.67:47163)
0619 10:20:34.998726 (+ 19us) negotiation.cc:107] Waiting for socket to connect
0619 10:20:34.998743 (+ 17us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.126:33511: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":12}
W20260619 10:20:34.998853 17667 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260619 10:20:35.002517 825 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTU9OP6/build/release/bin/kudu as pid 17538
I20260619 10:20:35.838503 17671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:20:35.838278 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:33511 (local address 127.0.206.65:43147)
0619 10:20:35.838401 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0619 10:20:35.838432 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.126:33511: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":94,"thread_start_us":56,"threads_started":1}
I20260619 10:20:35.873602 17672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:20:35.873416 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:33511 (local address 127.0.206.66:50491)
0619 10:20:35.873523 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0619 10:20:35.873556 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.126:33511: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":93,"thread_start_us":50,"threads_started":1}
I20260619 10:20:35.999300 17673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:20:35.999116 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:33511 (local address 127.0.206.67:44333)
0619 10:20:35.999213 (+ 97us) negotiation.cc:107] Waiting for socket to connect
0619 10:20:35.999244 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.126:33511: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":84,"thread_start_us":50,"threads_started":1}
I20260619 10:20:36.508046 17674 raft_consensus.cc:482] T ff30921c89da49ae958cbe5701678f1a P d835bbc06b3d47f7b10e69f31a298ed9 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 10:20:36.508224 17674 raft_consensus.cc:504] T ff30921c89da49ae958cbe5701678f1a P d835bbc06b3d47f7b10e69f31a298ed9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69870ff4ad6a44ecb692fb40428dec06" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 34349 } } peers { permanent_uuid: "c1b395dd05a64924a62063d847ac8bb2" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 38085 } } peers { permanent_uuid: "d835bbc06b3d47f7b10e69f31a298ed9" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 41351 } }
I20260619 10:20:36.508563 17674 leader_election.cc:288] T ff30921c89da49ae958cbe5701678f1a P d835bbc06b3d47f7b10e69f31a298ed9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 69870ff4ad6a44ecb692fb40428dec06 (127.0.206.66:34349), c1b395dd05a64924a62063d847ac8bb2 (127.0.206.67:38085)
Jun 19 10:20:36 dist-test-slave-6pt2 krb5kdc[16099](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864434, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.65@KRBTEST.COM for kudu/127.0.206.67@KRBTEST.COM
Jun 19 10:20:36 dist-test-slave-6pt2 krb5kdc[16099](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864434, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.65@KRBTEST.COM for kudu/127.0.206.66@KRBTEST.COM
I20260619 10:20:36.518378 17677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:20:36.508794 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.65:49625 (local address 127.0.206.66:34349)
0619 10:20:36.508980 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0619 10:20:36.508994 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0619 10:20:36.509020 (+ 26us) server_negotiation.cc:407] Connection header received
0619 10:20:36.509588 (+ 568us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:20:36.509591 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:20:36.509649 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:20:36.509814 (+ 165us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:20:36.510426 (+ 612us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:20:36.511304 (+ 878us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:20:36.512303 (+ 999us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:20:36.512688 (+ 385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:20:36.512796 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:20:36.515312 (+ 2516us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 10:20:36.515315 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 10:20:36.515316 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 10:20:36.515327 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0619 10:20:36.516865 (+ 1538us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:20:36.517279 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:20:36.517281 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:20:36.517282 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:20:36.517373 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:20:36.517576 (+ 203us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:20:36.517577 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:20:36.517578 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:20:36.517793 (+ 215us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 10:20:36.517880 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0619 10:20:36.518143 (+ 263us) server_negotiation.cc:299] Negotiation successful
0619 10:20:36.518258 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":123,"thread_start_us":63,"threads_started":1}
I20260619 10:20:36.518363 17675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:20:36.508695 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.66:34349 (local address 127.0.206.65:49625)
0619 10:20:36.508809 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0619 10:20:36.508819 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 10:20:36.508868 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 10:20:36.509853 (+ 985us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 10:20:36.509855 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 10:20:36.510157 (+ 302us) client_negotiation.cc:189] Negotiated authn=SASL
0619 10:20:36.510323 (+ 166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:20:36.510329 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:20:36.511393 (+ 1064us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:20:36.511394 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:20:36.512210 (+ 816us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:20:36.512213 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:20:36.512769 (+ 556us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:20:36.512770 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:20:36.512910 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:20:36.513422 (+ 512us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 10:20:36.513429 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0619 10:20:36.515218 (+ 1789us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 10:20:36.516958 (+ 1740us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 10:20:36.516959 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 10:20:36.516959 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 10:20:36.517162 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 10:20:36.517440 (+ 278us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 10:20:36.517441 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 10:20:36.517441 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 10:20:36.517514 (+ 73us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 10:20:36.517864 (+ 350us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 10:20:36.517870 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 10:20:36.517991 (+ 121us) client_negotiation.cc:769] Sending connection context
0619 10:20:36.518094 (+ 103us) client_negotiation.cc:240] Negotiation successful
0619 10:20:36.518269 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":101,"mutex_wait_us":43,"thread_start_us":64,"threads_started":1}
I20260619 10:20:36.518399 17678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:20:36.508902 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.65:44861 (local address 127.0.206.67:38085)
0619 10:20:36.509036 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0619 10:20:36.509044 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 10:20:36.509063 (+ 19us) server_negotiation.cc:407] Connection header received
0619 10:20:36.509588 (+ 525us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:20:36.509591 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:20:36.509649 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:20:36.509814 (+ 165us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:20:36.510449 (+ 635us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:20:36.511311 (+ 862us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:20:36.512320 (+ 1009us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:20:36.512692 (+ 372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:20:36.512807 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:20:36.514989 (+ 2182us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 10:20:36.514992 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 10:20:36.514992 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 10:20:36.515001 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0619 10:20:36.516788 (+ 1787us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:20:36.517216 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:20:36.517219 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:20:36.517220 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:20:36.517328 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:20:36.517564 (+ 236us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:20:36.517565 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:20:36.517566 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:20:36.517793 (+ 227us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 10:20:36.517880 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0619 10:20:36.518145 (+ 265us) server_negotiation.cc:299] Negotiation successful
0619 10:20:36.518263 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":97,"thread_start_us":55,"threads_started":1}
I20260619 10:20:36.518661 17676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:20:36.508700 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.67:38085 (local address 127.0.206.65:44861)
0619 10:20:36.508953 (+ 253us) negotiation.cc:107] Waiting for socket to connect
0619 10:20:36.508965 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 10:20:36.509021 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 10:20:36.509941 (+ 920us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 10:20:36.509941 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 10:20:36.510133 (+ 192us) client_negotiation.cc:189] Negotiated authn=SASL
0619 10:20:36.510326 (+ 193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:20:36.510332 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:20:36.511424 (+ 1092us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:20:36.511425 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:20:36.512209 (+ 784us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:20:36.512211 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:20:36.512783 (+ 572us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:20:36.512784 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:20:36.512907 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:20:36.513465 (+ 558us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 10:20:36.513469 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 10:20:36.514875 (+ 1406us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 10:20:36.516884 (+ 2009us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 10:20:36.516891 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 10:20:36.516892 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 10:20:36.517132 (+ 240us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 10:20:36.517394 (+ 262us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 10:20:36.517395 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 10:20:36.517395 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 10:20:36.517500 (+ 105us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 10:20:36.517862 (+ 362us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 10:20:36.517868 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 10:20:36.517991 (+ 123us) client_negotiation.cc:769] Sending connection context
0619 10:20:36.518094 (+ 103us) client_negotiation.cc:240] Negotiation successful
0619 10:20:36.518348 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":238,"mutex_wait_us":160,"thread_start_us":74,"threads_started":1}
W20260619 10:20:36.519326 17290 leader_election.cc:341] T ff30921c89da49ae958cbe5701678f1a P d835bbc06b3d47f7b10e69f31a298ed9 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer c1b395dd05a64924a62063d847ac8bb2 (127.0.206.67:38085): Not found: Tablet not found: ff30921c89da49ae958cbe5701678f1a
W20260619 10:20:36.519398 17288 leader_election.cc:341] T ff30921c89da49ae958cbe5701678f1a P d835bbc06b3d47f7b10e69f31a298ed9 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 69870ff4ad6a44ecb692fb40428dec06 (127.0.206.66:34349): Not found: Tablet not found: ff30921c89da49ae958cbe5701678f1a
I20260619 10:20:36.519454 17288 leader_election.cc:302] T ff30921c89da49ae958cbe5701678f1a P d835bbc06b3d47f7b10e69f31a298ed9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d835bbc06b3d47f7b10e69f31a298ed9; no voters: 69870ff4ad6a44ecb692fb40428dec06, c1b395dd05a64924a62063d847ac8bb2
I20260619 10:20:36.519541 17674 raft_consensus.cc:2726] T ff30921c89da49ae958cbe5701678f1a P d835bbc06b3d47f7b10e69f31a298ed9 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260619 10:20:36.839085 17676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:20:36.838970 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:33511 (local address 127.0.206.65:55727)
0619 10:20:36.839016 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0619 10:20:36.839047 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.126:33511: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
I20260619 10:20:36.874249 17679 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:20:36.873957 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:33511 (local address 127.0.206.66:57473)
0619 10:20:36.874173 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0619 10:20:36.874215 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.126:33511: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":201,"thread_start_us":62,"threads_started":1}
I20260619 10:20:37.000090 17680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:20:36.999758 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:33511 (local address 127.0.206.67:51291)
0619 10:20:37.000000 (+ 242us) negotiation.cc:107] Waiting for socket to connect
0619 10:20:37.000048 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.126:33511: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":223,"thread_start_us":92,"threads_started":1}
I20260619 10:20:37.839758 17681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:20:37.839504 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:33511 (local address 127.0.206.65:39943)
0619 10:20:37.839630 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0619 10:20:37.839694 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.126:33511: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":61,"threads_started":1}
I20260619 10:20:37.874933 17682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:20:37.874692 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:33511 (local address 127.0.206.66:41521)
0619 10:20:37.874848 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0619 10:20:37.874884 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.126:33511: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":83,"threads_started":1}
I20260619 10:20:38.000689 17683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:20:38.000495 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:33511 (local address 127.0.206.67:50969)
0619 10:20:38.000613 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0619 10:20:38.000651 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.126:33511: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":56,"threads_started":1}
I20260619 10:20:38.294788 17684 raft_consensus.cc:482] T ff30921c89da49ae958cbe5701678f1a P d835bbc06b3d47f7b10e69f31a298ed9 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 10:20:38.294878 17684 raft_consensus.cc:504] T ff30921c89da49ae958cbe5701678f1a P d835bbc06b3d47f7b10e69f31a298ed9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69870ff4ad6a44ecb692fb40428dec06" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 34349 } } peers { permanent_uuid: "c1b395dd05a64924a62063d847ac8bb2" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 38085 } } peers { permanent_uuid: "d835bbc06b3d47f7b10e69f31a298ed9" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 41351 } }
I20260619 10:20:38.295030 17684 leader_election.cc:288] T ff30921c89da49ae958cbe5701678f1a P d835bbc06b3d47f7b10e69f31a298ed9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 69870ff4ad6a44ecb692fb40428dec06 (127.0.206.66:34349), c1b395dd05a64924a62063d847ac8bb2 (127.0.206.67:38085)
W20260619 10:20:38.295423 17288 leader_election.cc:341] T ff30921c89da49ae958cbe5701678f1a P d835bbc06b3d47f7b10e69f31a298ed9 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 69870ff4ad6a44ecb692fb40428dec06 (127.0.206.66:34349): Not found: Tablet not found: ff30921c89da49ae958cbe5701678f1a
W20260619 10:20:38.295502 17290 leader_election.cc:341] T ff30921c89da49ae958cbe5701678f1a P d835bbc06b3d47f7b10e69f31a298ed9 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer c1b395dd05a64924a62063d847ac8bb2 (127.0.206.67:38085): Not found: Tablet not found: ff30921c89da49ae958cbe5701678f1a
I20260619 10:20:38.295573 17290 leader_election.cc:302] T ff30921c89da49ae958cbe5701678f1a P d835bbc06b3d47f7b10e69f31a298ed9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d835bbc06b3d47f7b10e69f31a298ed9; no voters: 69870ff4ad6a44ecb692fb40428dec06, c1b395dd05a64924a62063d847ac8bb2
I20260619 10:20:38.295652 17684 raft_consensus.cc:2726] T ff30921c89da49ae958cbe5701678f1a P d835bbc06b3d47f7b10e69f31a298ed9 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260619 10:20:38.840596 17685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:20:38.840351 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:33511 (local address 127.0.206.65:40323)
0619 10:20:38.840492 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0619 10:20:38.840552 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.126:33511: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":83,"threads_started":1}
I20260619 10:20:38.875524 17686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:20:38.875308 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:33511 (local address 127.0.206.66:42493)
0619 10:20:38.875440 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0619 10:20:38.875471 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.126:33511: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":70,"threads_started":1}
I20260619 10:20:39.001327 17687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:20:39.001144 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:33511 (local address 127.0.206.67:34563)
0619 10:20:39.001257 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0619 10:20:39.001290 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.126:33511: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":98,"thread_start_us":57,"threads_started":1}
I20260619 10:20:39.841218 17688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:20:39.841032 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:33511 (local address 127.0.206.65:34373)
0619 10:20:39.841144 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0619 10:20:39.841176 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.126:33511: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":99,"thread_start_us":56,"threads_started":1}
I20260619 10:20:39.876107 17689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:20:39.875928 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:33511 (local address 127.0.206.66:36675)
0619 10:20:39.876046 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0619 10:20:39.876078 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.126:33511: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":78,"threads_started":1}
I20260619 10:20:39.903725 17690 raft_consensus.cc:482] T ff30921c89da49ae958cbe5701678f1a P d835bbc06b3d47f7b10e69f31a298ed9 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 10:20:39.903812 17690 raft_consensus.cc:504] T ff30921c89da49ae958cbe5701678f1a P d835bbc06b3d47f7b10e69f31a298ed9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69870ff4ad6a44ecb692fb40428dec06" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 34349 } } peers { permanent_uuid: "c1b395dd05a64924a62063d847ac8bb2" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 38085 } } peers { permanent_uuid: "d835bbc06b3d47f7b10e69f31a298ed9" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 41351 } }
I20260619 10:20:39.904000 17690 leader_election.cc:288] T ff30921c89da49ae958cbe5701678f1a P d835bbc06b3d47f7b10e69f31a298ed9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 69870ff4ad6a44ecb692fb40428dec06 (127.0.206.66:34349), c1b395dd05a64924a62063d847ac8bb2 (127.0.206.67:38085)
W20260619 10:20:39.904389 17290 leader_election.cc:341] T ff30921c89da49ae958cbe5701678f1a P d835bbc06b3d47f7b10e69f31a298ed9 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer c1b395dd05a64924a62063d847ac8bb2 (127.0.206.67:38085): Not found: Tablet not found: ff30921c89da49ae958cbe5701678f1a
W20260619 10:20:39.904469 17288 leader_election.cc:341] T ff30921c89da49ae958cbe5701678f1a P d835bbc06b3d47f7b10e69f31a298ed9 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 69870ff4ad6a44ecb692fb40428dec06 (127.0.206.66:34349): Not found: Tablet not found: ff30921c89da49ae958cbe5701678f1a
I20260619 10:20:39.904512 17288 leader_election.cc:302] T ff30921c89da49ae958cbe5701678f1a P d835bbc06b3d47f7b10e69f31a298ed9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d835bbc06b3d47f7b10e69f31a298ed9; no voters: 69870ff4ad6a44ecb692fb40428dec06, c1b395dd05a64924a62063d847ac8bb2
I20260619 10:20:39.904598 17690 raft_consensus.cc:2726] T ff30921c89da49ae958cbe5701678f1a P d835bbc06b3d47f7b10e69f31a298ed9 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260619 10:20:40.002135 17691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:20:40.001842 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:33511 (local address 127.0.206.67:55643)
0619 10:20:40.002066 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0619 10:20:40.002101 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.126:33511: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":208,"thread_start_us":50,"threads_started":1}
I20260619 10:20:40.841897 17692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:20:40.841613 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:33511 (local address 127.0.206.65:58645)
0619 10:20:40.841801 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0619 10:20:40.841836 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.126:33511: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":171,"thread_start_us":53,"threads_started":1}
I20260619 10:20:40.876684 17693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:20:40.876512 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:33511 (local address 127.0.206.66:44045)
0619 10:20:40.876619 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0619 10:20:40.876649 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.126:33511: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":93,"thread_start_us":56,"threads_started":1}
I20260619 10:20:41.002760 17694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:20:41.002531 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:33511 (local address 127.0.206.67:40677)
0619 10:20:41.002688 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0619 10:20:41.002721 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.126:33511: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":102,"threads_started":1}
I20260619 10:20:41.842504 17695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:20:41.842307 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:33511 (local address 127.0.206.65:53621)
0619 10:20:41.842425 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0619 10:20:41.842456 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.126:33511: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":60,"threads_started":1}
I20260619 10:20:41.877328 17696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:20:41.877072 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:33511 (local address 127.0.206.66:38719)
0619 10:20:41.877242 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0619 10:20:41.877292 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.126:33511: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":84,"threads_started":1}
I20260619 10:20:42.003319 17697 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:20:42.003137 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:33511 (local address 127.0.206.67:56145)
0619 10:20:42.003244 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0619 10:20:42.003277 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.126:33511: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":94,"thread_start_us":55,"threads_started":1}
I20260619 10:20:42.843149 17698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:20:42.842937 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:33511 (local address 127.0.206.65:48295)
0619 10:20:42.843062 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0619 10:20:42.843101 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.126:33511: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":58,"threads_started":1}
I20260619 10:20:42.877871 17699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:20:42.877700 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:33511 (local address 127.0.206.66:60201)
0619 10:20:42.877803 (+ 103us) negotiation.cc:107] Waiting for socket to connect
0619 10:20:42.877833 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.126:33511: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":89,"thread_start_us":54,"threads_started":1}
I20260619 10:20:43.003901 17700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:20:43.003699 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:33511 (local address 127.0.206.67:34753)
0619 10:20:43.003825 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0619 10:20:43.003860 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.126:33511: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":70,"threads_started":1}
I20260619 10:20:43.843799 17701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:20:43.843578 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:33511 (local address 127.0.206.65:48343)
0619 10:20:43.843718 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0619 10:20:43.843753 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.126:33511: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":59,"threads_started":1}
I20260619 10:20:43.878479 17702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:20:43.878276 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:33511 (local address 127.0.206.66:46109)
0619 10:20:43.878398 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0619 10:20:43.878431 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.126:33511: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":62,"threads_started":1}
I20260619 10:20:44.004495 17703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:20:44.004297 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:33511 (local address 127.0.206.67:56665)
0619 10:20:44.004423 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0619 10:20:44.004456 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.126:33511: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":69,"threads_started":1}
I20260619 10:20:44.458590 17704 raft_consensus.cc:482] T ff30921c89da49ae958cbe5701678f1a P d835bbc06b3d47f7b10e69f31a298ed9 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 10:20:44.458838 17704 raft_consensus.cc:504] T ff30921c89da49ae958cbe5701678f1a P d835bbc06b3d47f7b10e69f31a298ed9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69870ff4ad6a44ecb692fb40428dec06" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 34349 } } peers { permanent_uuid: "c1b395dd05a64924a62063d847ac8bb2" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 38085 } } peers { permanent_uuid: "d835bbc06b3d47f7b10e69f31a298ed9" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 41351 } }
I20260619 10:20:44.459040 17704 leader_election.cc:288] T ff30921c89da49ae958cbe5701678f1a P d835bbc06b3d47f7b10e69f31a298ed9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 69870ff4ad6a44ecb692fb40428dec06 (127.0.206.66:34349), c1b395dd05a64924a62063d847ac8bb2 (127.0.206.67:38085)
W20260619 10:20:44.459474 17288 leader_election.cc:341] T ff30921c89da49ae958cbe5701678f1a P d835bbc06b3d47f7b10e69f31a298ed9 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 69870ff4ad6a44ecb692fb40428dec06 (127.0.206.66:34349): Not found: Tablet not found: ff30921c89da49ae958cbe5701678f1a
W20260619 10:20:44.459571 17290 leader_election.cc:341] T ff30921c89da49ae958cbe5701678f1a P d835bbc06b3d47f7b10e69f31a298ed9 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer c1b395dd05a64924a62063d847ac8bb2 (127.0.206.67:38085): Not found: Tablet not found: ff30921c89da49ae958cbe5701678f1a
I20260619 10:20:44.459617 17290 leader_election.cc:302] T ff30921c89da49ae958cbe5701678f1a P d835bbc06b3d47f7b10e69f31a298ed9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d835bbc06b3d47f7b10e69f31a298ed9; no voters: 69870ff4ad6a44ecb692fb40428dec06, c1b395dd05a64924a62063d847ac8bb2
I20260619 10:20:44.459702 17704 raft_consensus.cc:2726] T ff30921c89da49ae958cbe5701678f1a P d835bbc06b3d47f7b10e69f31a298ed9 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260619 10:20:44.844720 17705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:20:44.844457 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:33511 (local address 127.0.206.65:58175)
0619 10:20:44.844626 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0619 10:20:44.844672 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.126:33511: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":81,"threads_started":1}
I20260619 10:20:44.879171 17706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:20:44.878952 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:33511 (local address 127.0.206.66:47127)
0619 10:20:44.879078 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0619 10:20:44.879118 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.126:33511: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":69,"threads_started":1}
I20260619 10:20:45.005295 17707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:20:45.005073 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:33511 (local address 127.0.206.67:46445)
0619 10:20:45.005193 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0619 10:20:45.005237 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.126:33511: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":55,"threads_started":1}
I20260619 10:20:45.845592 17708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:20:45.845352 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:33511 (local address 127.0.206.65:40511)
0619 10:20:45.845508 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0619 10:20:45.845545 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.126:33511: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":70,"threads_started":1}
I20260619 10:20:45.879796 17709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:20:45.879600 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:33511 (local address 127.0.206.66:41117)
0619 10:20:45.879731 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0619 10:20:45.879758 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.126:33511: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":88,"threads_started":1}
I20260619 10:20:46.005939 17710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:20:46.005743 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:33511 (local address 127.0.206.67:53195)
0619 10:20:46.005857 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0619 10:20:46.005894 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.126:33511: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":99,"thread_start_us":57,"threads_started":1}
I20260619 10:20:46.846305 17711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:20:46.846096 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:33511 (local address 127.0.206.65:36635)
0619 10:20:46.846223 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0619 10:20:46.846258 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.126:33511: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":69,"threads_started":1}
I20260619 10:20:46.880388 17712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:20:46.880215 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:33511 (local address 127.0.206.66:53035)
0619 10:20:46.880314 (+ 99us) negotiation.cc:107] Waiting for socket to connect
0619 10:20:46.880336 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.126:33511: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":88,"thread_start_us":49,"threads_started":1}
I20260619 10:20:47.006584 17713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:20:47.006400 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:33511 (local address 127.0.206.67:43511)
0619 10:20:47.006510 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0619 10:20:47.006544 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.126:33511: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":95,"thread_start_us":56,"threads_started":1}
I20260619 10:20:47.847059 17714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:20:47.846813 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:33511 (local address 127.0.206.65:57681)
0619 10:20:47.846972 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0619 10:20:47.847014 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.126:33511: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":63,"threads_started":1}
I20260619 10:20:47.880992 17715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:20:47.880783 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:33511 (local address 127.0.206.66:35149)
0619 10:20:47.880921 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0619 10:20:47.880954 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.126:33511: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":70,"threads_started":1}
I20260619 10:20:48.007264 17716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:20:48.007063 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:33511 (local address 127.0.206.67:33397)
0619 10:20:48.007189 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0619 10:20:48.007223 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.126:33511: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":58,"threads_started":1}
I20260619 10:20:48.847715 17717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:20:48.847477 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:33511 (local address 127.0.206.65:56935)
0619 10:20:48.847620 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0619 10:20:48.847661 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.126:33511: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":62,"threads_started":1}
I20260619 10:20:48.881633 17718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:20:48.881439 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:33511 (local address 127.0.206.66:40987)
0619 10:20:48.881562 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0619 10:20:48.881593 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.126:33511: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":70,"threads_started":1}
I20260619 10:20:49.007915 17719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:20:49.007701 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:33511 (local address 127.0.206.67:52815)
0619 10:20:49.007828 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0619 10:20:49.007864 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.126:33511: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":71,"threads_started":1}
I20260619 10:20:49.848546 17720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:20:49.848346 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:33511 (local address 127.0.206.65:56251)
0619 10:20:49.848470 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0619 10:20:49.848503 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.126:33511: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":59,"threads_started":1}
I20260619 10:20:49.882310 17721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:20:49.882141 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:33511 (local address 127.0.206.66:50397)
0619 10:20:49.882249 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0619 10:20:49.882278 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.126:33511: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":95,"thread_start_us":58,"threads_started":1}
I20260619 10:20:50.008072 825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTU9OP6/build/release/bin/kudu with pid 17240
I20260619 10:20:50.009361 17722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:20:50.008704 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:33511 (local address 127.0.206.67:36975)
0619 10:20:50.009251 (+ 547us) negotiation.cc:107] Waiting for socket to connect
0619 10:20:50.009287 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.126:33511: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":529,"thread_start_us":64,"threads_started":1}
I20260619 10:20:50.012764 825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTU9OP6/build/release/bin/kudu with pid 17405
I20260619 10:20:50.016980 825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTU9OP6/build/release/bin/kudu with pid 17538
2026-06-19T10:20:50Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (58333 ms)
[----------] 30 tests from SecurityITest (225447 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781864224577058-825-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 10:20:50 dist-test-slave-6pt2 krb5kdc[17725](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 10:20:50 dist-test-slave-6pt2 krb5kdc[17725](info): set up 2 sockets
Jun 19 10:20:50 dist-test-slave-6pt2 krb5kdc[17725](info): commencing operation
krb5kdc: starting...
W20260619 10:20:51.030871 825 mini_kdc.cc:121] Time spent starting KDC: real 0.994s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781864224577058-825-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781864224577058-825-0/krb5kdc/test-user.keytab.
Jun 19 10:20:51 dist-test-slave-6pt2 krb5kdc[17725](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864451, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T10:20:51Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T10:20:51Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.206.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.206.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.206.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781864224577058-825-0/krb5kdc/kudu_127.0.206.126.keytab.
Entry for principal kudu/127.0.206.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781864224577058-825-0/krb5kdc/kudu_127.0.206.126.keytab.
I20260619 10:20:51.140663 825 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTU9OP6/build/release/bin/kudu
/tmp/dist-test-taskTU9OP6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781864224577058-825-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781864224577058-825-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781864224577058-825-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781864224577058-825-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.206.126:32875
--webserver_interface=127.0.206.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781864224577058-825-0/krb5kdc/kudu_127.0.206.126.keytab
--principal=kudu/127.0.206.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.206.84:33565
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.206.126:32875
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781864224577058-825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781864224577058-825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781864224577058-825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 10:20:51.218123 17738 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 10:20:51.218346 17738 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 10:20:51.218381 17738 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 10:20:51.219492 17738 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 10:20:51.219558 17738 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 10:20:51.219581 17738 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 10:20:51.219602 17738 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 10:20:51.219630 17738 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 10:20:51.220968 17738 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.206.84:33565
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781864224577058-825-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781864224577058-825-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.206.126:32875
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781864224577058-825-0/krb5kdc/kudu_127.0.206.126.keytab
--principal=kudu/127.0.206.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.206.126:32875
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781864224577058-825-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.206.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17738
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781864224577058-825-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 09:32:44 UTC on ccycloud.quasar-gbtldp.root.comops.site
build id 4455440
I20260619 10:20:51.221222 17738 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 10:20:51.221465 17738 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 10:20:51.226387 17744 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 10:20:51.226477 17746 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 10:20:51.226493 17743 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 10:20:51.226485 17738 server_base.cc:1034] running on GCE node
I20260619 10:20:51.227061 17738 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 10:20:51.227281 17738 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 10:20:51.228418 17738 hybrid_clock.cc:648] HybridClock initialized: now 1781864451228411 us; error 30 us; skew 500 ppm
Jun 19 10:20:51 dist-test-slave-6pt2 krb5kdc[17725](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864451, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 10:20:51.230696 17738 init.cc:377] Logged in from keytab as kudu/127.0.206.126@KRBTEST.COM (short username kudu)
I20260619 10:20:51.231132 17738 webserver.cc:460] Webserver started at http://127.0.206.126:46049/ using document root <none> and password file <none>
I20260619 10:20:51.231357 17738 fs_manager.cc:359] Metadata directory not provided
I20260619 10:20:51.231431 17738 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 10:20:51.231595 17738 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 10:20:51.232547 17738 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781864224577058-825-0/minicluster-data/master-0/data/instance:
uuid: "f3199a9fe74449d3aa7ca50e8a02ccd0"
format_stamp: "Formatted at 2026-06-19 10:20:51 on dist-test-slave-6pt2"
server_key: "e2d3bce38226a16e1d5cbababa829faa"
server_key_iv: "edf0181efbf4c58d0b79b489c7a7f8dc"
server_key_version: "encryptionkey@0"
I20260619 10:20:51.232898 17738 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781864224577058-825-0/minicluster-data/master-0/wal/instance:
uuid: "f3199a9fe74449d3aa7ca50e8a02ccd0"
format_stamp: "Formatted at 2026-06-19 10:20:51 on dist-test-slave-6pt2"
server_key: "e2d3bce38226a16e1d5cbababa829faa"
server_key_iv: "edf0181efbf4c58d0b79b489c7a7f8dc"
server_key_version: "encryptionkey@0"
I20260619 10:20:51.234177 17738 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260619 10:20:51.234961 17753 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 10:20:51.235118 17738 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260619 10:20:51.235214 17738 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781864224577058-825-0/minicluster-data/master-0/data,/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781864224577058-825-0/minicluster-data/master-0/wal
uuid: "f3199a9fe74449d3aa7ca50e8a02ccd0"
format_stamp: "Formatted at 2026-06-19 10:20:51 on dist-test-slave-6pt2"
server_key: "e2d3bce38226a16e1d5cbababa829faa"
server_key_iv: "edf0181efbf4c58d0b79b489c7a7f8dc"
server_key_version: "encryptionkey@0"
I20260619 10:20:51.235282 17738 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781864224577058-825-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781864224577058-825-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781864224577058-825-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 10:20:51.261775 17738 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 10:20:51.263535 17738 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 10:20:51.263823 17738 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 10:20:51.267757 17738 rpc_server.cc:307] RPC server started. Bound to: 127.0.206.126:32875
I20260619 10:20:51.267787 17805 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.206.126:32875 every 8 connection(s)
I20260619 10:20:51.268108 17738 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781864224577058-825-0/minicluster-data/master-0/data/info.pb
I20260619 10:20:51.268929 17806 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 10:20:51.271018 17806 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f3199a9fe74449d3aa7ca50e8a02ccd0: Bootstrap starting.
I20260619 10:20:51.271488 17806 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f3199a9fe74449d3aa7ca50e8a02ccd0: Neither blocks nor log segments found. Creating new log.
I20260619 10:20:51.271741 17806 log.cc:824] T 00000000000000000000000000000000 P f3199a9fe74449d3aa7ca50e8a02ccd0: Log is configured to *not* fsync() on all Append() calls
I20260619 10:20:51.272347 17806 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f3199a9fe74449d3aa7ca50e8a02ccd0: No bootstrap required, opened a new log
I20260619 10:20:51.273803 17806 raft_consensus.cc:348] T 00000000000000000000000000000000 P f3199a9fe74449d3aa7ca50e8a02ccd0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f3199a9fe74449d3aa7ca50e8a02ccd0" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 32875 } }
I20260619 10:20:51.273936 17806 raft_consensus.cc:374] T 00000000000000000000000000000000 P f3199a9fe74449d3aa7ca50e8a02ccd0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 10:20:51.273958 17806 raft_consensus.cc:729] T 00000000000000000000000000000000 P f3199a9fe74449d3aa7ca50e8a02ccd0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f3199a9fe74449d3aa7ca50e8a02ccd0, State: Initialized, Role: FOLLOWER
I20260619 10:20:51.274050 17806 consensus_queue.cc:260] T 00000000000000000000000000000000 P f3199a9fe74449d3aa7ca50e8a02ccd0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f3199a9fe74449d3aa7ca50e8a02ccd0" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 32875 } }
I20260619 10:20:51.274130 17806 raft_consensus.cc:388] T 00000000000000000000000000000000 P f3199a9fe74449d3aa7ca50e8a02ccd0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 10:20:51.274163 17806 raft_consensus.cc:482] T 00000000000000000000000000000000 P f3199a9fe74449d3aa7ca50e8a02ccd0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 10:20:51.274231 17806 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f3199a9fe74449d3aa7ca50e8a02ccd0 [term 0 FOLLOWER]: Advancing to term 1
I20260619 10:20:51.274750 17806 raft_consensus.cc:504] T 00000000000000000000000000000000 P f3199a9fe74449d3aa7ca50e8a02ccd0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f3199a9fe74449d3aa7ca50e8a02ccd0" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 32875 } }
I20260619 10:20:51.274880 17806 leader_election.cc:302] T 00000000000000000000000000000000 P f3199a9fe74449d3aa7ca50e8a02ccd0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f3199a9fe74449d3aa7ca50e8a02ccd0; no voters:
I20260619 10:20:51.275049 17806 leader_election.cc:288] T 00000000000000000000000000000000 P f3199a9fe74449d3aa7ca50e8a02ccd0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 10:20:51.275053 825 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTU9OP6/build/release/bin/kudu as pid 17738
I20260619 10:20:51.275098 17809 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f3199a9fe74449d3aa7ca50e8a02ccd0 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 10:20:51.275179 825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781864224577058-825-0/minicluster-data/master-0/wal/instance
I20260619 10:20:51.275204 17809 raft_consensus.cc:686] T 00000000000000000000000000000000 P f3199a9fe74449d3aa7ca50e8a02ccd0 [term 1 LEADER]: Becoming Leader. State: Replica: f3199a9fe74449d3aa7ca50e8a02ccd0, State: Running, Role: LEADER
I20260619 10:20:51.275265 17809 consensus_queue.cc:237] T 00000000000000000000000000000000 P f3199a9fe74449d3aa7ca50e8a02ccd0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f3199a9fe74449d3aa7ca50e8a02ccd0" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 32875 } }
I20260619 10:20:51.275298 17806 sys_catalog.cc:564] T 00000000000000000000000000000000 P f3199a9fe74449d3aa7ca50e8a02ccd0 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 10:20:51.275369 825 external_mini_cluster.cc:1428] Setting key c8f996c9a80c8b443776909090a8b580
I20260619 10:20:51.275661 17811 sys_catalog.cc:455] T 00000000000000000000000000000000 P f3199a9fe74449d3aa7ca50e8a02ccd0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f3199a9fe74449d3aa7ca50e8a02ccd0. Latest consensus state: current_term: 1 leader_uuid: "f3199a9fe74449d3aa7ca50e8a02ccd0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f3199a9fe74449d3aa7ca50e8a02ccd0" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 32875 } } }
I20260619 10:20:51.275769 17810 sys_catalog.cc:455] T 00000000000000000000000000000000 P f3199a9fe74449d3aa7ca50e8a02ccd0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f3199a9fe74449d3aa7ca50e8a02ccd0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f3199a9fe74449d3aa7ca50e8a02ccd0" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 32875 } } }
I20260619 10:20:51.275835 17811 sys_catalog.cc:458] T 00000000000000000000000000000000 P f3199a9fe74449d3aa7ca50e8a02ccd0 [sys.catalog]: This master's current role is: LEADER
I20260619 10:20:51.275843 17810 sys_catalog.cc:458] T 00000000000000000000000000000000 P f3199a9fe74449d3aa7ca50e8a02ccd0 [sys.catalog]: This master's current role is: LEADER
I20260619 10:20:51.276147 17819 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 10:20:51.276937 17819 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 10:20:51.279178 17819 catalog_manager.cc:1347] Generated new cluster ID: 0c047d765b8841b3997452e7c9df998c
I20260619 10:20:51.279266 17819 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Jun 19 10:20:51 dist-test-slave-6pt2 krb5kdc[17725](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864451, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.206.126@KRBTEST.COM
I20260619 10:20:51.286351 17819 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 10:20:51.286995 17819 catalog_manager.cc:1504] Loading token signing keys...
I20260619 10:20:51.287294 17820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:20:51.276082 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43936 (local address 127.0.206.126:32875)
0619 10:20:51.276260 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0619 10:20:51.276325 (+ 65us) server_negotiation.cc:399] Waiting for connection header
0619 10:20:51.276685 (+ 360us) server_negotiation.cc:407] Connection header received
0619 10:20:51.276808 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:20:51.276815 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:20:51.276866 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:20:51.277032 (+ 166us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:20:51.277886 (+ 854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:20:51.279039 (+ 1153us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:20:51.280076 (+ 1037us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:20:51.280706 (+ 630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:20:51.280820 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:20:51.283544 (+ 2724us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 10:20:51.283548 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 10:20:51.283548 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 10:20:51.283559 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0619 10:20:51.285443 (+ 1884us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:20:51.285819 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:20:51.285822 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:20:51.285822 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:20:51.285918 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:20:51.286151 (+ 233us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:20:51.286152 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:20:51.286152 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:20:51.286407 (+ 255us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 10:20:51.286550 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0619 10:20:51.286956 (+ 406us) server_negotiation.cc:299] Negotiation successful
0619 10:20:51.287090 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":120,"thread_start_us":60,"threads_started":1}
I20260619 10:20:51.289990 17819 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f3199a9fe74449d3aa7ca50e8a02ccd0: Generated new TSK 0
I20260619 10:20:51.290119 17819 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.0.206.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.206.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.206.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781864224577058-825-0/krb5kdc/kudu_127.0.206.65.keytab.
Entry for principal kudu/127.0.206.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781864224577058-825-0/krb5kdc/kudu_127.0.206.65.keytab.
I20260619 10:20:51.315291 825 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTU9OP6/build/release/bin/kudu
/tmp/dist-test-taskTU9OP6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781864224577058-825-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781864224577058-825-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781864224577058-825-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781864224577058-825-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.206.65:0
--local_ip_for_outbound_sockets=127.0.206.65
--webserver_interface=127.0.206.65
--webserver_port=0
--tserver_master_addrs=127.0.206.126:32875
--keytab_file=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781864224577058-825-0/krb5kdc/kudu_127.0.206.65.keytab
--principal=kudu/127.0.206.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.206.84:33565
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781864224577058-825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781864224577058-825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781864224577058-825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 10:20:51.397872 17832 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 10:20:51.398067 17832 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 10:20:51.398090 17832 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 10:20:51.399302 17832 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 10:20:51.399340 17832 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 10:20:51.399365 17832 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.206.65
I20260619 10:20:51.400928 17832 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.206.84:33565
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781864224577058-825-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781864224577058-825-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781864224577058-825-0/krb5kdc/kudu_127.0.206.65.keytab
--principal=kudu/127.0.206.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.206.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781864224577058-825-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.206.65
--webserver_port=0
--tserver_master_addrs=127.0.206.126:32875
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17832
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.206.65
--log_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781864224577058-825-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 09:32:44 UTC on ccycloud.quasar-gbtldp.root.comops.site
build id 4455440
I20260619 10:20:51.401139 17832 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 10:20:51.401384 17832 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 10:20:51.406051 17838 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 10:20:51.406103 17840 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 10:20:51.406234 17832 server_base.cc:1034] running on GCE node
W20260619 10:20:51.406108 17837 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 10:20:51.406533 17832 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 10:20:51.406800 17832 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 10:20:51.407971 17832 hybrid_clock.cc:648] HybridClock initialized: now 1781864451407964 us; error 35 us; skew 500 ppm
Jun 19 10:20:51 dist-test-slave-6pt2 krb5kdc[17725](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864451, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 10:20:51.410423 17832 init.cc:377] Logged in from keytab as kudu/127.0.206.65@KRBTEST.COM (short username kudu)
I20260619 10:20:51.410848 17832 webserver.cc:460] Webserver started at http://127.0.206.65:42259/ using document root <none> and password file <none>
I20260619 10:20:51.411059 17832 fs_manager.cc:359] Metadata directory not provided
I20260619 10:20:51.411125 17832 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 10:20:51.411269 17832 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 10:20:51.412241 17832 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781864224577058-825-0/minicluster-data/ts-0/data/instance:
uuid: "1671860ad7cd49a5a9acfa9ebd4b8794"
format_stamp: "Formatted at 2026-06-19 10:20:51 on dist-test-slave-6pt2"
server_key: "155efd76630919627c0852f89044e4e0"
server_key_iv: "b19629e511db0ab387f6fe10e785f37e"
server_key_version: "encryptionkey@0"
I20260619 10:20:51.412591 17832 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781864224577058-825-0/minicluster-data/ts-0/wal/instance:
uuid: "1671860ad7cd49a5a9acfa9ebd4b8794"
format_stamp: "Formatted at 2026-06-19 10:20:51 on dist-test-slave-6pt2"
server_key: "155efd76630919627c0852f89044e4e0"
server_key_iv: "b19629e511db0ab387f6fe10e785f37e"
server_key_version: "encryptionkey@0"
I20260619 10:20:51.413862 17832 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 10:20:51.414561 17847 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 10:20:51.414754 17832 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 10:20:51.414855 17832 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781864224577058-825-0/minicluster-data/ts-0/data,/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781864224577058-825-0/minicluster-data/ts-0/wal
uuid: "1671860ad7cd49a5a9acfa9ebd4b8794"
format_stamp: "Formatted at 2026-06-19 10:20:51 on dist-test-slave-6pt2"
server_key: "155efd76630919627c0852f89044e4e0"
server_key_iv: "b19629e511db0ab387f6fe10e785f37e"
server_key_version: "encryptionkey@0"
I20260619 10:20:51.414935 17832 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781864224577058-825-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781864224577058-825-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781864224577058-825-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 10:20:51.441396 17832 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 10:20:51.443822 17832 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 10:20:51.443984 17832 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 10:20:51.444213 17832 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 10:20:51.444521 17832 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 10:20:51.444578 17832 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 10:20:51.444619 17832 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 10:20:51.444648 17832 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 10:20:51.450268 17832 rpc_server.cc:307] RPC server started. Bound to: 127.0.206.65:45317
I20260619 10:20:51.450341 17960 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.206.65:45317 every 8 connection(s)
I20260619 10:20:51.450678 17832 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781864224577058-825-0/minicluster-data/ts-0/data/info.pb
I20260619 10:20:51.451679 825 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTU9OP6/build/release/bin/kudu as pid 17832
I20260619 10:20:51.451808 825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781864224577058-825-0/minicluster-data/ts-0/wal/instance
I20260619 10:20:51.452008 825 external_mini_cluster.cc:1428] Setting key 3f74d75c49233348562278d2ba6ececa
Jun 19 10:20:51 dist-test-slave-6pt2 krb5kdc[17725](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864451, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.65@KRBTEST.COM for kudu/127.0.206.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.0.206.66@KRBTEST.COM; defaulting to no policy
I20260619 10:20:51.460964 17820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:20:51.451418 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.65:48823 (local address 127.0.206.126:32875)
0619 10:20:51.451481 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0619 10:20:51.451483 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 10:20:51.451653 (+ 170us) server_negotiation.cc:407] Connection header received
0619 10:20:51.452866 (+ 1213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:20:51.452867 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:20:51.452880 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:20:51.452953 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:20:51.453570 (+ 617us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:20:51.454347 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:20:51.455650 (+ 1303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:20:51.455837 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:20:51.455917 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:20:51.458228 (+ 2311us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 10:20:51.458230 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 10:20:51.458230 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 10:20:51.458238 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 10:20:51.459606 (+ 1368us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:20:51.459999 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:20:51.460000 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:20:51.460001 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:20:51.460059 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:20:51.460298 (+ 239us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:20:51.460299 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:20:51.460300 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:20:51.460412 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 10:20:51.460484 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0619 10:20:51.460768 (+ 284us) server_negotiation.cc:299] Negotiation successful
0619 10:20:51.460858 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260619 10:20:51.461124 17963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:20:51.451375 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:32875 (local address 127.0.206.65:48823)
0619 10:20:51.451524 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0619 10:20:51.451564 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0619 10:20:51.452749 (+ 1185us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 10:20:51.453020 (+ 271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 10:20:51.453023 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 10:20:51.453266 (+ 243us) client_negotiation.cc:189] Negotiated authn=SASL
0619 10:20:51.453473 (+ 207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:20:51.453478 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:20:51.454436 (+ 958us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:20:51.454438 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:20:51.455572 (+ 1134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:20:51.455575 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:20:51.455927 (+ 352us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:20:51.455928 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:20:51.456053 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:20:51.456522 (+ 469us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 10:20:51.456530 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 10:20:51.458133 (+ 1603us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 10:20:51.459702 (+ 1569us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 10:20:51.459708 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 10:20:51.459710 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 10:20:51.459924 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 10:20:51.460132 (+ 208us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 10:20:51.460133 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 10:20:51.460133 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 10:20:51.460234 (+ 101us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 10:20:51.460487 (+ 253us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 10:20:51.460494 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 10:20:51.460625 (+ 131us) client_negotiation.cc:769] Sending connection context
0619 10:20:51.460744 (+ 119us) client_negotiation.cc:240] Negotiation successful
0619 10:20:51.460897 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":85,"threads_started":1}
I20260619 10:20:51.461637 17961 heartbeater.cc:347] Connected to a master server at 127.0.206.126:32875
I20260619 10:20:51.461742 17961 heartbeater.cc:464] Registering TS with master...
I20260619 10:20:51.461913 17961 heartbeater.cc:511] Master 127.0.206.126:32875 requested a full tablet report, sending...
I20260619 10:20:51.462447 17770 ts_manager.cc:194] Registered new tserver with Master: 1671860ad7cd49a5a9acfa9ebd4b8794 (127.0.206.65:45317)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.206.66@KRBTEST.COM" created.
I20260619 10:20:51.463240 17770 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.206.65@KRBTEST.COM'} at 127.0.206.65:48823
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.206.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781864224577058-825-0/krb5kdc/kudu_127.0.206.66.keytab.
Entry for principal kudu/127.0.206.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781864224577058-825-0/krb5kdc/kudu_127.0.206.66.keytab.
I20260619 10:20:51.475203 825 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTU9OP6/build/release/bin/kudu
/tmp/dist-test-taskTU9OP6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781864224577058-825-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781864224577058-825-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781864224577058-825-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781864224577058-825-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.206.66:0
--local_ip_for_outbound_sockets=127.0.206.66
--webserver_interface=127.0.206.66
--webserver_port=0
--tserver_master_addrs=127.0.206.126:32875
--keytab_file=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781864224577058-825-0/krb5kdc/kudu_127.0.206.66.keytab
--principal=kudu/127.0.206.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.206.84:33565
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781864224577058-825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781864224577058-825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781864224577058-825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 10:20:51.553803 17966 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 10:20:51.554028 17966 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 10:20:51.554062 17966 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 10:20:51.555194 17966 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 10:20:51.555264 17966 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 10:20:51.555315 17966 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.206.66
I20260619 10:20:51.556727 17966 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.206.84:33565
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781864224577058-825-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781864224577058-825-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781864224577058-825-0/krb5kdc/kudu_127.0.206.66.keytab
--principal=kudu/127.0.206.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.206.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781864224577058-825-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.206.66
--webserver_port=0
--tserver_master_addrs=127.0.206.126:32875
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17966
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.206.66
--log_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781864224577058-825-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 09:32:44 UTC on ccycloud.quasar-gbtldp.root.comops.site
build id 4455440
I20260619 10:20:51.556973 17966 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 10:20:51.557195 17966 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 10:20:51.561908 17974 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 10:20:51.561981 17972 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 10:20:51.562001 17966 server_base.cc:1034] running on GCE node
W20260619 10:20:51.561916 17971 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 10:20:51.562379 17966 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 10:20:51.562678 17966 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 10:20:51.563841 17966 hybrid_clock.cc:648] HybridClock initialized: now 1781864451563830 us; error 25 us; skew 500 ppm
Jun 19 10:20:51 dist-test-slave-6pt2 krb5kdc[17725](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864451, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 10:20:51.566219 17966 init.cc:377] Logged in from keytab as kudu/127.0.206.66@KRBTEST.COM (short username kudu)
I20260619 10:20:51.566653 17966 webserver.cc:460] Webserver started at http://127.0.206.66:46501/ using document root <none> and password file <none>
I20260619 10:20:51.566881 17966 fs_manager.cc:359] Metadata directory not provided
I20260619 10:20:51.566968 17966 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 10:20:51.567098 17966 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 10:20:51.568135 17966 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781864224577058-825-0/minicluster-data/ts-1/data/instance:
uuid: "77f4d396ff12475abe32edf4a915c40d"
format_stamp: "Formatted at 2026-06-19 10:20:51 on dist-test-slave-6pt2"
server_key: "8dfb25f07853cf6fa989a8c4fdd13be1"
server_key_iv: "e9a024ce736e55667f80b53c9b59d3f7"
server_key_version: "encryptionkey@0"
I20260619 10:20:51.568481 17966 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781864224577058-825-0/minicluster-data/ts-1/wal/instance:
uuid: "77f4d396ff12475abe32edf4a915c40d"
format_stamp: "Formatted at 2026-06-19 10:20:51 on dist-test-slave-6pt2"
server_key: "8dfb25f07853cf6fa989a8c4fdd13be1"
server_key_iv: "e9a024ce736e55667f80b53c9b59d3f7"
server_key_version: "encryptionkey@0"
I20260619 10:20:51.569787 17966 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260619 10:20:51.570452 17981 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 10:20:51.570662 17966 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260619 10:20:51.570747 17966 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781864224577058-825-0/minicluster-data/ts-1/data,/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781864224577058-825-0/minicluster-data/ts-1/wal
uuid: "77f4d396ff12475abe32edf4a915c40d"
format_stamp: "Formatted at 2026-06-19 10:20:51 on dist-test-slave-6pt2"
server_key: "8dfb25f07853cf6fa989a8c4fdd13be1"
server_key_iv: "e9a024ce736e55667f80b53c9b59d3f7"
server_key_version: "encryptionkey@0"
I20260619 10:20:51.570830 17966 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781864224577058-825-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781864224577058-825-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781864224577058-825-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 10:20:51.582577 17966 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 10:20:51.584254 17966 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 10:20:51.584407 17966 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 10:20:51.584620 17966 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 10:20:51.584903 17966 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 10:20:51.584955 17966 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 10:20:51.584988 17966 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 10:20:51.585034 17966 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 10:20:51.590588 17966 rpc_server.cc:307] RPC server started. Bound to: 127.0.206.66:45237
I20260619 10:20:51.590660 18094 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.206.66:45237 every 8 connection(s)
I20260619 10:20:51.590931 17966 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781864224577058-825-0/minicluster-data/ts-1/data/info.pb
Jun 19 10:20:51 dist-test-slave-6pt2 krb5kdc[17725](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864451, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.66@KRBTEST.COM for kudu/127.0.206.126@KRBTEST.COM
I20260619 10:20:51.599363 17820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:20:51.591540 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.66:39883 (local address 127.0.206.126:32875)
0619 10:20:51.591602 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0619 10:20:51.591604 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 10:20:51.591780 (+ 176us) server_negotiation.cc:407] Connection header received
0619 10:20:51.592686 (+ 906us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:20:51.592688 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:20:51.592701 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:20:51.592755 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:20:51.593340 (+ 585us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:20:51.593840 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:20:51.594919 (+ 1079us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:20:51.595130 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:20:51.595214 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:20:51.597125 (+ 1911us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 10:20:51.597126 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 10:20:51.597127 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 10:20:51.597133 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 10:20:51.598244 (+ 1111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:20:51.598639 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:20:51.598639 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:20:51.598640 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:20:51.598693 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:20:51.598902 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:20:51.598903 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:20:51.598903 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:20:51.598996 (+ 93us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 10:20:51.599042 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0619 10:20:51.599246 (+ 204us) server_negotiation.cc:299] Negotiation successful
0619 10:20:51.599303 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260619 10:20:51.599424 825 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTU9OP6/build/release/bin/kudu as pid 17966
I20260619 10:20:51.599529 825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781864224577058-825-0/minicluster-data/ts-1/wal/instance
I20260619 10:20:51.599676 18097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:20:51.591492 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:32875 (local address 127.0.206.66:39883)
0619 10:20:51.591669 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0619 10:20:51.591700 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0619 10:20:51.592580 (+ 880us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 10:20:51.592815 (+ 235us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 10:20:51.592818 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 10:20:51.593081 (+ 263us) client_negotiation.cc:189] Negotiated authn=SASL
0619 10:20:51.593247 (+ 166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:20:51.593251 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:20:51.593950 (+ 699us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:20:51.593951 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:20:51.594830 (+ 879us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:20:51.594833 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:20:51.595224 (+ 391us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:20:51.595225 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:20:51.595351 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:20:51.595782 (+ 431us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 10:20:51.595789 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0619 10:20:51.597041 (+ 1252us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 10:20:51.598311 (+ 1270us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 10:20:51.598316 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 10:20:51.598318 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 10:20:51.598522 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 10:20:51.598748 (+ 226us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 10:20:51.598749 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 10:20:51.598749 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 10:20:51.598839 (+ 90us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 10:20:51.599048 (+ 209us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 10:20:51.599051 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 10:20:51.599148 (+ 97us) client_negotiation.cc:769] Sending connection context
0619 10:20:51.599222 (+ 74us) client_negotiation.cc:240] Negotiation successful
0619 10:20:51.599417 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":75,"threads_started":1}
I20260619 10:20:51.599754 825 external_mini_cluster.cc:1428] Setting key a7d10fda5279e54583a382eed7fb11cb
I20260619 10:20:51.600236 18095 heartbeater.cc:347] Connected to a master server at 127.0.206.126:32875
I20260619 10:20:51.600325 18095 heartbeater.cc:464] Registering TS with master...
I20260619 10:20:51.600490 18095 heartbeater.cc:511] Master 127.0.206.126:32875 requested a full tablet report, sending...
I20260619 10:20:51.600868 17769 ts_manager.cc:194] Registered new tserver with Master: 77f4d396ff12475abe32edf4a915c40d (127.0.206.66:45237)
I20260619 10:20:51.601243 17769 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.206.66@KRBTEST.COM'} at 127.0.206.66:39883
WARNING: no policy specified for kudu/127.0.206.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.206.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.206.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781864224577058-825-0/krb5kdc/kudu_127.0.206.67.keytab.
Entry for principal kudu/127.0.206.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781864224577058-825-0/krb5kdc/kudu_127.0.206.67.keytab.
I20260619 10:20:51.623134 825 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTU9OP6/build/release/bin/kudu
/tmp/dist-test-taskTU9OP6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781864224577058-825-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781864224577058-825-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781864224577058-825-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781864224577058-825-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.206.67:0
--local_ip_for_outbound_sockets=127.0.206.67
--webserver_interface=127.0.206.67
--webserver_port=0
--tserver_master_addrs=127.0.206.126:32875
--keytab_file=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781864224577058-825-0/krb5kdc/kudu_127.0.206.67.keytab
--principal=kudu/127.0.206.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.206.84:33565
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781864224577058-825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781864224577058-825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781864224577058-825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 10:20:51.701545 18100 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 10:20:51.701745 18100 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 10:20:51.701767 18100 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 10:20:51.702999 18100 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 10:20:51.703038 18100 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 10:20:51.703064 18100 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.206.67
I20260619 10:20:51.704494 18100 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.206.84:33565
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781864224577058-825-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781864224577058-825-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781864224577058-825-0/krb5kdc/kudu_127.0.206.67.keytab
--principal=kudu/127.0.206.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.206.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781864224577058-825-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.206.67
--webserver_port=0
--tserver_master_addrs=127.0.206.126:32875
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18100
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.206.67
--log_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781864224577058-825-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 09:32:44 UTC on ccycloud.quasar-gbtldp.root.comops.site
build id 4455440
I20260619 10:20:51.704713 18100 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 10:20:51.704939 18100 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 10:20:51.709640 18108 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 10:20:51.709717 18106 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 10:20:51.709640 18105 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 10:20:51.709743 18100 server_base.cc:1034] running on GCE node
I20260619 10:20:51.710115 18100 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 10:20:51.710342 18100 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 10:20:51.711508 18100 hybrid_clock.cc:648] HybridClock initialized: now 1781864451711493 us; error 29 us; skew 500 ppm
Jun 19 10:20:51 dist-test-slave-6pt2 krb5kdc[17725](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864451, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 10:20:51.714092 18100 init.cc:377] Logged in from keytab as kudu/127.0.206.67@KRBTEST.COM (short username kudu)
I20260619 10:20:51.714488 18100 webserver.cc:460] Webserver started at http://127.0.206.67:45465/ using document root <none> and password file <none>
I20260619 10:20:51.714752 18100 fs_manager.cc:359] Metadata directory not provided
I20260619 10:20:51.714820 18100 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 10:20:51.714977 18100 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 10:20:51.715950 18100 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781864224577058-825-0/minicluster-data/ts-2/data/instance:
uuid: "5072f8b972474ce7bfd2d8da646f16d5"
format_stamp: "Formatted at 2026-06-19 10:20:51 on dist-test-slave-6pt2"
server_key: "e9c799d0d5e6d14e33462153d50e280c"
server_key_iv: "98c129f83b1aadef65c86a705a352394"
server_key_version: "encryptionkey@0"
I20260619 10:20:51.716298 18100 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781864224577058-825-0/minicluster-data/ts-2/wal/instance:
uuid: "5072f8b972474ce7bfd2d8da646f16d5"
format_stamp: "Formatted at 2026-06-19 10:20:51 on dist-test-slave-6pt2"
server_key: "e9c799d0d5e6d14e33462153d50e280c"
server_key_iv: "98c129f83b1aadef65c86a705a352394"
server_key_version: "encryptionkey@0"
I20260619 10:20:51.717587 18100 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 10:20:51.718288 18115 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 10:20:51.718463 18100 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 10:20:51.718557 18100 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781864224577058-825-0/minicluster-data/ts-2/data,/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781864224577058-825-0/minicluster-data/ts-2/wal
uuid: "5072f8b972474ce7bfd2d8da646f16d5"
format_stamp: "Formatted at 2026-06-19 10:20:51 on dist-test-slave-6pt2"
server_key: "e9c799d0d5e6d14e33462153d50e280c"
server_key_iv: "98c129f83b1aadef65c86a705a352394"
server_key_version: "encryptionkey@0"
I20260619 10:20:51.718678 18100 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781864224577058-825-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781864224577058-825-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781864224577058-825-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 10:20:51.739977 18100 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 10:20:51.742177 18100 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 10:20:51.742329 18100 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 10:20:51.742704 18100 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 10:20:51.743062 18100 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 10:20:51.743114 18100 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 10:20:51.743153 18100 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 10:20:51.743180 18100 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 10:20:51.748878 18100 rpc_server.cc:307] RPC server started. Bound to: 127.0.206.67:35557
I20260619 10:20:51.748919 18228 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.206.67:35557 every 8 connection(s)
I20260619 10:20:51.749187 18100 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781864224577058-825-0/minicluster-data/ts-2/data/info.pb
Jun 19 10:20:51 dist-test-slave-6pt2 krb5kdc[17725](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864451, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.67@KRBTEST.COM for kudu/127.0.206.126@KRBTEST.COM
I20260619 10:20:51.757356 825 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTU9OP6/build/release/bin/kudu as pid 18100
I20260619 10:20:51.757455 825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781864224577058-825-0/minicluster-data/ts-2/wal/instance
I20260619 10:20:51.757656 825 external_mini_cluster.cc:1428] Setting key c3edb3faffccfb64196c0b79ff240226
I20260619 10:20:51.758371 17820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:20:51.749808 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.67:60137 (local address 127.0.206.126:32875)
0619 10:20:51.749887 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0619 10:20:51.749890 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 10:20:51.750091 (+ 201us) server_negotiation.cc:407] Connection header received
0619 10:20:51.751116 (+ 1025us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:20:51.751118 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:20:51.751131 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:20:51.751181 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:20:51.751765 (+ 584us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:20:51.752270 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:20:51.753262 (+ 992us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:20:51.753465 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:20:51.753547 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:20:51.755647 (+ 2100us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 10:20:51.755649 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 10:20:51.755649 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 10:20:51.755655 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 10:20:51.756982 (+ 1327us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:20:51.757331 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:20:51.757332 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:20:51.757333 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:20:51.757392 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:20:51.757644 (+ 252us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:20:51.757645 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:20:51.757645 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:20:51.757780 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 10:20:51.757868 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0619 10:20:51.758175 (+ 307us) server_negotiation.cc:299] Negotiation successful
0619 10:20:51.758256 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260619 10:20:51.758757 18231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:20:51.749782 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:32875 (local address 127.0.206.67:60137)
0619 10:20:51.749964 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0619 10:20:51.750005 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0619 10:20:51.751004 (+ 999us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 10:20:51.751249 (+ 245us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 10:20:51.751253 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 10:20:51.751501 (+ 248us) client_negotiation.cc:189] Negotiated authn=SASL
0619 10:20:51.751675 (+ 174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:20:51.751679 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:20:51.752361 (+ 682us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:20:51.752362 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:20:51.753160 (+ 798us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:20:51.753162 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:20:51.753556 (+ 394us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:20:51.753558 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:20:51.753674 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:20:51.754133 (+ 459us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 10:20:51.754142 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0619 10:20:51.755563 (+ 1421us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 10:20:51.757049 (+ 1486us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 10:20:51.757053 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 10:20:51.757055 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 10:20:51.757249 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 10:20:51.757466 (+ 217us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 10:20:51.757467 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 10:20:51.757468 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 10:20:51.757578 (+ 110us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 10:20:51.757868 (+ 290us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 10:20:51.757875 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 10:20:51.758019 (+ 144us) client_negotiation.cc:769] Sending connection context
0619 10:20:51.758142 (+ 123us) client_negotiation.cc:240] Negotiation successful
0619 10:20:51.758324 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":81,"threads_started":1}
I20260619 10:20:51.759133 18229 heartbeater.cc:347] Connected to a master server at 127.0.206.126:32875
I20260619 10:20:51.759235 18229 heartbeater.cc:464] Registering TS with master...
I20260619 10:20:51.759402 18229 heartbeater.cc:511] Master 127.0.206.126:32875 requested a full tablet report, sending...
I20260619 10:20:51.759706 17769 ts_manager.cc:194] Registered new tserver with Master: 5072f8b972474ce7bfd2d8da646f16d5 (127.0.206.67:35557)
I20260619 10:20:51.760041 17769 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.206.67@KRBTEST.COM'} at 127.0.206.67:60137
I20260619 10:20:51.769551 825 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 10:20:51.777046 17820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:20:51.770918 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60398 (local address 127.0.206.126:32875)
0619 10:20:51.770987 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0619 10:20:51.770989 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 10:20:51.771096 (+ 107us) server_negotiation.cc:407] Connection header received
0619 10:20:51.771171 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:20:51.771172 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:20:51.771181 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:20:51.771247 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:20:51.771800 (+ 553us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:20:51.772266 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:20:51.773036 (+ 770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:20:51.773262 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:20:51.773331 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:20:51.774455 (+ 1124us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 10:20:51.774456 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 10:20:51.774456 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 10:20:51.774462 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 10:20:51.775971 (+ 1509us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:20:51.776272 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:20:51.776273 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:20:51.776274 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:20:51.776319 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:20:51.776530 (+ 211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:20:51.776531 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:20:51.776531 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:20:51.776664 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 10:20:51.776763 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0619 10:20:51.776874 (+ 111us) server_negotiation.cc:299] Negotiation successful
0619 10:20:51.776952 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260619 10:20:51.778678 17769 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60398:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 10:20:51.780098 17769 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 10:20:51.786232 18240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:20:51.782076 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46706 (local address 127.0.206.67:35557)
0619 10:20:51.782226 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0619 10:20:51.782230 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 10:20:51.782519 (+ 289us) server_negotiation.cc:407] Connection header received
0619 10:20:51.782665 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:20:51.782668 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:20:51.782684 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:20:51.782768 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 10:20:51.783040 (+ 272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:20:51.783729 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:20:51.785140 (+ 1411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:20:51.785718 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:20:51.785783 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:20:51.785845 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0619 10:20:51.786097 (+ 252us) server_negotiation.cc:299] Negotiation successful
0619 10:20:51.786156 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":122,"thread_start_us":73,"threads_started":1}
I20260619 10:20:51.786487 18238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:20:51.782053 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.65:45317 (local address 127.0.0.1:60208)
0619 10:20:51.782422 (+ 369us) negotiation.cc:107] Waiting for socket to connect
0619 10:20:51.782462 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0619 10:20:51.782537 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 10:20:51.782868 (+ 331us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 10:20:51.782869 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 10:20:51.782870 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 10:20:51.782920 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:20:51.782922 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:20:51.784243 (+ 1321us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:20:51.784244 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:20:51.785467 (+ 1223us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:20:51.785470 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:20:51.786156 (+ 686us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:20:51.786157 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:20:51.786219 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:20:51.786222 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 10:20:51.786276 (+ 54us) client_negotiation.cc:240] Negotiation successful
0619 10:20:51.786345 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":355,"mutex_wait_us":1,"thread_start_us":44,"threads_started":1}
I20260619 10:20:51.786648 18241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:20:51.782242 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60208 (local address 127.0.206.65:45317)
0619 10:20:51.782410 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0619 10:20:51.782413 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 10:20:51.782536 (+ 123us) server_negotiation.cc:407] Connection header received
0619 10:20:51.782666 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:20:51.782668 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:20:51.782699 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:20:51.782795 (+ 96us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 10:20:51.783027 (+ 232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:20:51.784129 (+ 1102us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:20:51.785536 (+ 1407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:20:51.786030 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:20:51.786145 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:20:51.786199 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0619 10:20:51.786476 (+ 277us) server_negotiation.cc:299] Negotiation successful
0619 10:20:51.786529 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":132,"thread_start_us":103,"threads_started":1}
I20260619 10:20:51.786731 18237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:20:51.782053 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.66:45237 (local address 127.0.0.1:40492)
0619 10:20:51.782663 (+ 610us) negotiation.cc:107] Waiting for socket to connect
0619 10:20:51.782671 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0619 10:20:51.782711 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 10:20:51.782991 (+ 280us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 10:20:51.782992 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 10:20:51.782994 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 10:20:51.783054 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:20:51.783055 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:20:51.784282 (+ 1227us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:20:51.784283 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:20:51.785408 (+ 1125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:20:51.785411 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:20:51.786471 (+ 1060us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:20:51.786472 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:20:51.786536 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:20:51.786540 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 10:20:51.786635 (+ 95us) client_negotiation.cc:240] Negotiation successful
0619 10:20:51.786671 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":602,"mutex_wait_us":97,"thread_start_us":47,"threads_started":1}
I20260619 10:20:51.786748 18242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:20:51.782242 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40492 (local address 127.0.206.66:45237)
0619 10:20:51.782477 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0619 10:20:51.782480 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 10:20:51.782766 (+ 286us) server_negotiation.cc:407] Connection header received
0619 10:20:51.782809 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:20:51.782812 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:20:51.782827 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:20:51.782896 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 10:20:51.783135 (+ 239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:20:51.784197 (+ 1062us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:20:51.785491 (+ 1294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:20:51.786256 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:20:51.786389 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:20:51.786446 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0619 10:20:51.786633 (+ 187us) server_negotiation.cc:299] Negotiation successful
0619 10:20:51.786670 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":205,"thread_start_us":89,"threads_started":1}
I20260619 10:20:51.787468 17895 tablet_service.cc:1467] Processing CreateTablet for tablet ef1e7d0183084efb9857bd039f690dcf (DEFAULT_TABLE table=test-table [id=f3f0d13da0814e3fb07610a6d680e004]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 10:20:51.787683 18029 tablet_service.cc:1467] Processing CreateTablet for tablet ef1e7d0183084efb9857bd039f690dcf (DEFAULT_TABLE table=test-table [id=f3f0d13da0814e3fb07610a6d680e004]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 10:20:51.787813 17895 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ef1e7d0183084efb9857bd039f690dcf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 10:20:51.787973 18029 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ef1e7d0183084efb9857bd039f690dcf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 10:20:51.786489 18239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:20:51.782012 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.67:35557 (local address 127.0.0.1:46706)
0619 10:20:51.782422 (+ 410us) negotiation.cc:107] Waiting for socket to connect
0619 10:20:51.782462 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0619 10:20:51.782537 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 10:20:51.782856 (+ 319us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 10:20:51.782859 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 10:20:51.782862 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 10:20:51.782929 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:20:51.782931 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:20:51.783830 (+ 899us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:20:51.783831 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:20:51.785049 (+ 1218us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:20:51.785051 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:20:51.785877 (+ 826us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:20:51.785878 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:20:51.785953 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:20:51.785963 (+ 10us) client_negotiation.cc:769] Sending connection context
0619 10:20:51.786320 (+ 357us) client_negotiation.cc:240] Negotiation successful
0619 10:20:51.786377 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":384,"mutex_wait_us":114,"thread_start_us":48,"threads_started":1}
I20260619 10:20:51.791028 18243 tablet_bootstrap.cc:492] T ef1e7d0183084efb9857bd039f690dcf P 77f4d396ff12475abe32edf4a915c40d: Bootstrap starting.
I20260619 10:20:51.791373 18163 tablet_service.cc:1467] Processing CreateTablet for tablet ef1e7d0183084efb9857bd039f690dcf (DEFAULT_TABLE table=test-table [id=f3f0d13da0814e3fb07610a6d680e004]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 10:20:51.791666 18163 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ef1e7d0183084efb9857bd039f690dcf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 10:20:51.791702 18243 tablet_bootstrap.cc:654] T ef1e7d0183084efb9857bd039f690dcf P 77f4d396ff12475abe32edf4a915c40d: Neither blocks nor log segments found. Creating new log.
I20260619 10:20:51.792214 18243 log.cc:824] T ef1e7d0183084efb9857bd039f690dcf P 77f4d396ff12475abe32edf4a915c40d: Log is configured to *not* fsync() on all Append() calls
I20260619 10:20:51.792565 18244 tablet_bootstrap.cc:492] T ef1e7d0183084efb9857bd039f690dcf P 1671860ad7cd49a5a9acfa9ebd4b8794: Bootstrap starting.
I20260619 10:20:51.792919 18243 tablet_bootstrap.cc:492] T ef1e7d0183084efb9857bd039f690dcf P 77f4d396ff12475abe32edf4a915c40d: No bootstrap required, opened a new log
I20260619 10:20:51.793017 18243 ts_tablet_manager.cc:1397] T ef1e7d0183084efb9857bd039f690dcf P 77f4d396ff12475abe32edf4a915c40d: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260619 10:20:51.793221 18244 tablet_bootstrap.cc:654] T ef1e7d0183084efb9857bd039f690dcf P 1671860ad7cd49a5a9acfa9ebd4b8794: Neither blocks nor log segments found. Creating new log.
I20260619 10:20:51.793754 18244 log.cc:824] T ef1e7d0183084efb9857bd039f690dcf P 1671860ad7cd49a5a9acfa9ebd4b8794: Log is configured to *not* fsync() on all Append() calls
I20260619 10:20:51.794514 18244 tablet_bootstrap.cc:492] T ef1e7d0183084efb9857bd039f690dcf P 1671860ad7cd49a5a9acfa9ebd4b8794: No bootstrap required, opened a new log
I20260619 10:20:51.794584 18244 ts_tablet_manager.cc:1397] T ef1e7d0183084efb9857bd039f690dcf P 1671860ad7cd49a5a9acfa9ebd4b8794: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260619 10:20:51.794739 18243 raft_consensus.cc:348] T ef1e7d0183084efb9857bd039f690dcf P 77f4d396ff12475abe32edf4a915c40d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5072f8b972474ce7bfd2d8da646f16d5" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 35557 } } peers { permanent_uuid: "77f4d396ff12475abe32edf4a915c40d" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 45237 } } peers { permanent_uuid: "1671860ad7cd49a5a9acfa9ebd4b8794" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 45317 } }
I20260619 10:20:51.794891 18243 raft_consensus.cc:374] T ef1e7d0183084efb9857bd039f690dcf P 77f4d396ff12475abe32edf4a915c40d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 10:20:51.794951 18243 raft_consensus.cc:729] T ef1e7d0183084efb9857bd039f690dcf P 77f4d396ff12475abe32edf4a915c40d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 77f4d396ff12475abe32edf4a915c40d, State: Initialized, Role: FOLLOWER
I20260619 10:20:51.795082 18243 consensus_queue.cc:260] T ef1e7d0183084efb9857bd039f690dcf P 77f4d396ff12475abe32edf4a915c40d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5072f8b972474ce7bfd2d8da646f16d5" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 35557 } } peers { permanent_uuid: "77f4d396ff12475abe32edf4a915c40d" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 45237 } } peers { permanent_uuid: "1671860ad7cd49a5a9acfa9ebd4b8794" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 45317 } }
I20260619 10:20:51.795373 18095 heartbeater.cc:503] Master 127.0.206.126:32875 was elected leader, sending a full tablet report...
I20260619 10:20:51.795370 18243 ts_tablet_manager.cc:1428] T ef1e7d0183084efb9857bd039f690dcf P 77f4d396ff12475abe32edf4a915c40d: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260619 10:20:51.795554 18247 tablet_bootstrap.cc:492] T ef1e7d0183084efb9857bd039f690dcf P 5072f8b972474ce7bfd2d8da646f16d5: Bootstrap starting.
I20260619 10:20:51.796059 18247 tablet_bootstrap.cc:654] T ef1e7d0183084efb9857bd039f690dcf P 5072f8b972474ce7bfd2d8da646f16d5: Neither blocks nor log segments found. Creating new log.
I20260619 10:20:51.796518 18247 log.cc:824] T ef1e7d0183084efb9857bd039f690dcf P 5072f8b972474ce7bfd2d8da646f16d5: Log is configured to *not* fsync() on all Append() calls
I20260619 10:20:51.796654 18244 raft_consensus.cc:348] T ef1e7d0183084efb9857bd039f690dcf P 1671860ad7cd49a5a9acfa9ebd4b8794 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5072f8b972474ce7bfd2d8da646f16d5" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 35557 } } peers { permanent_uuid: "77f4d396ff12475abe32edf4a915c40d" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 45237 } } peers { permanent_uuid: "1671860ad7cd49a5a9acfa9ebd4b8794" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 45317 } }
I20260619 10:20:51.796826 18244 raft_consensus.cc:374] T ef1e7d0183084efb9857bd039f690dcf P 1671860ad7cd49a5a9acfa9ebd4b8794 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 10:20:51.796871 18244 raft_consensus.cc:729] T ef1e7d0183084efb9857bd039f690dcf P 1671860ad7cd49a5a9acfa9ebd4b8794 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1671860ad7cd49a5a9acfa9ebd4b8794, State: Initialized, Role: FOLLOWER
I20260619 10:20:51.796975 18244 consensus_queue.cc:260] T ef1e7d0183084efb9857bd039f690dcf P 1671860ad7cd49a5a9acfa9ebd4b8794 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5072f8b972474ce7bfd2d8da646f16d5" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 35557 } } peers { permanent_uuid: "77f4d396ff12475abe32edf4a915c40d" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 45237 } } peers { permanent_uuid: "1671860ad7cd49a5a9acfa9ebd4b8794" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 45317 } }
I20260619 10:20:51.797247 18244 ts_tablet_manager.cc:1428] T ef1e7d0183084efb9857bd039f690dcf P 1671860ad7cd49a5a9acfa9ebd4b8794: Time spent starting tablet: real 0.003s user 0.000s sys 0.002s
I20260619 10:20:51.797307 18247 tablet_bootstrap.cc:492] T ef1e7d0183084efb9857bd039f690dcf P 5072f8b972474ce7bfd2d8da646f16d5: No bootstrap required, opened a new log
I20260619 10:20:51.797277 17961 heartbeater.cc:503] Master 127.0.206.126:32875 was elected leader, sending a full tablet report...
I20260619 10:20:51.797362 18247 ts_tablet_manager.cc:1397] T ef1e7d0183084efb9857bd039f690dcf P 5072f8b972474ce7bfd2d8da646f16d5: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260619 10:20:51.798969 18247 raft_consensus.cc:348] T ef1e7d0183084efb9857bd039f690dcf P 5072f8b972474ce7bfd2d8da646f16d5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5072f8b972474ce7bfd2d8da646f16d5" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 35557 } } peers { permanent_uuid: "77f4d396ff12475abe32edf4a915c40d" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 45237 } } peers { permanent_uuid: "1671860ad7cd49a5a9acfa9ebd4b8794" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 45317 } }
I20260619 10:20:51.799113 18247 raft_consensus.cc:374] T ef1e7d0183084efb9857bd039f690dcf P 5072f8b972474ce7bfd2d8da646f16d5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 10:20:51.799157 18247 raft_consensus.cc:729] T ef1e7d0183084efb9857bd039f690dcf P 5072f8b972474ce7bfd2d8da646f16d5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5072f8b972474ce7bfd2d8da646f16d5, State: Initialized, Role: FOLLOWER
I20260619 10:20:51.799280 18247 consensus_queue.cc:260] T ef1e7d0183084efb9857bd039f690dcf P 5072f8b972474ce7bfd2d8da646f16d5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5072f8b972474ce7bfd2d8da646f16d5" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 35557 } } peers { permanent_uuid: "77f4d396ff12475abe32edf4a915c40d" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 45237 } } peers { permanent_uuid: "1671860ad7cd49a5a9acfa9ebd4b8794" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 45317 } }
I20260619 10:20:51.799563 18229 heartbeater.cc:503] Master 127.0.206.126:32875 was elected leader, sending a full tablet report...
I20260619 10:20:51.799574 18247 ts_tablet_manager.cc:1428] T ef1e7d0183084efb9857bd039f690dcf P 5072f8b972474ce7bfd2d8da646f16d5: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260619 10:20:51.822345 18248 raft_consensus.cc:482] T ef1e7d0183084efb9857bd039f690dcf P 77f4d396ff12475abe32edf4a915c40d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 10:20:51.822443 18248 raft_consensus.cc:504] T ef1e7d0183084efb9857bd039f690dcf P 77f4d396ff12475abe32edf4a915c40d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5072f8b972474ce7bfd2d8da646f16d5" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 35557 } } peers { permanent_uuid: "77f4d396ff12475abe32edf4a915c40d" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 45237 } } peers { permanent_uuid: "1671860ad7cd49a5a9acfa9ebd4b8794" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 45317 } }
I20260619 10:20:51.822876 18248 leader_election.cc:288] T ef1e7d0183084efb9857bd039f690dcf P 77f4d396ff12475abe32edf4a915c40d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5072f8b972474ce7bfd2d8da646f16d5 (127.0.206.67:35557), 1671860ad7cd49a5a9acfa9ebd4b8794 (127.0.206.65:45317)
I20260619 10:20:51.826666 18241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:20:51.823257 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.66:58621 (local address 127.0.206.65:45317)
0619 10:20:51.823300 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0619 10:20:51.823301 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 10:20:51.823311 (+ 10us) server_negotiation.cc:407] Connection header received
0619 10:20:51.823332 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:20:51.823340 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:20:51.823355 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:20:51.823396 (+ 41us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 10:20:51.823553 (+ 157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:20:51.824224 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:20:51.825723 (+ 1499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:20:51.826300 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:20:51.826377 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:20:51.826384 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0619 10:20:51.826545 (+ 161us) server_negotiation.cc:299] Negotiation successful
0619 10:20:51.826582 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260619 10:20:51.826666 18240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:20:51.823080 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.66:51197 (local address 127.0.206.67:35557)
0619 10:20:51.823134 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0619 10:20:51.823136 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 10:20:51.823144 (+ 8us) server_negotiation.cc:407] Connection header received
0619 10:20:51.823167 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:20:51.823168 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:20:51.823179 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:20:51.823253 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 10:20:51.823454 (+ 201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:20:51.824223 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:20:51.825722 (+ 1499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:20:51.826300 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:20:51.826376 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:20:51.826384 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0619 10:20:51.826545 (+ 161us) server_negotiation.cc:299] Negotiation successful
0619 10:20:51.826581 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260619 10:20:51.826681 18097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:20:51.823004 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.67:35557 (local address 127.0.206.66:51197)
0619 10:20:51.823047 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0619 10:20:51.823060 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 10:20:51.823102 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 10:20:51.823279 (+ 177us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 10:20:51.823280 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 10:20:51.823282 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 10:20:51.823355 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:20:51.823357 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:20:51.824311 (+ 954us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:20:51.824312 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:20:51.825630 (+ 1318us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:20:51.825633 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:20:51.826384 (+ 751us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:20:51.826385 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:20:51.826471 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:20:51.826474 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 10:20:51.826525 (+ 51us) client_negotiation.cc:240] Negotiation successful
0619 10:20:51.826560 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":22,"mutex_wait_us":60}
I20260619 10:20:51.826687 18252 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:20:51.823037 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.65:45317 (local address 127.0.206.66:58621)
0619 10:20:51.823214 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0619 10:20:51.823224 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 10:20:51.823278 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 10:20:51.823420 (+ 142us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 10:20:51.823420 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 10:20:51.823422 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 10:20:51.823488 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:20:51.823489 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:20:51.824311 (+ 822us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:20:51.824312 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:20:51.825630 (+ 1318us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:20:51.825633 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:20:51.826391 (+ 758us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:20:51.826391 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:20:51.826459 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:20:51.826463 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 10:20:51.826516 (+ 53us) client_negotiation.cc:240] Negotiation successful
0619 10:20:51.826558 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":157,"mutex_wait_us":66,"thread_start_us":95,"threads_started":1}
I20260619 10:20:51.827121 17915 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ef1e7d0183084efb9857bd039f690dcf" candidate_uuid: "77f4d396ff12475abe32edf4a915c40d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1671860ad7cd49a5a9acfa9ebd4b8794" is_pre_election: true
I20260619 10:20:51.827121 18183 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ef1e7d0183084efb9857bd039f690dcf" candidate_uuid: "77f4d396ff12475abe32edf4a915c40d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5072f8b972474ce7bfd2d8da646f16d5" is_pre_election: true
I20260619 10:20:51.827263 18183 raft_consensus.cc:2454] T ef1e7d0183084efb9857bd039f690dcf P 5072f8b972474ce7bfd2d8da646f16d5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 77f4d396ff12475abe32edf4a915c40d in term 0.
I20260619 10:20:51.827263 17915 raft_consensus.cc:2454] T ef1e7d0183084efb9857bd039f690dcf P 1671860ad7cd49a5a9acfa9ebd4b8794 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 77f4d396ff12475abe32edf4a915c40d in term 0.
I20260619 10:20:51.827483 17982 leader_election.cc:302] T ef1e7d0183084efb9857bd039f690dcf P 77f4d396ff12475abe32edf4a915c40d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1671860ad7cd49a5a9acfa9ebd4b8794, 77f4d396ff12475abe32edf4a915c40d; no voters:
I20260619 10:20:51.827629 18248 raft_consensus.cc:2781] T ef1e7d0183084efb9857bd039f690dcf P 77f4d396ff12475abe32edf4a915c40d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 10:20:51.827682 18248 raft_consensus.cc:482] T ef1e7d0183084efb9857bd039f690dcf P 77f4d396ff12475abe32edf4a915c40d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 10:20:51.827709 18248 raft_consensus.cc:3037] T ef1e7d0183084efb9857bd039f690dcf P 77f4d396ff12475abe32edf4a915c40d [term 0 FOLLOWER]: Advancing to term 1
I20260619 10:20:51.828390 18248 raft_consensus.cc:504] T ef1e7d0183084efb9857bd039f690dcf P 77f4d396ff12475abe32edf4a915c40d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5072f8b972474ce7bfd2d8da646f16d5" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 35557 } } peers { permanent_uuid: "77f4d396ff12475abe32edf4a915c40d" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 45237 } } peers { permanent_uuid: "1671860ad7cd49a5a9acfa9ebd4b8794" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 45317 } }
I20260619 10:20:51.828524 18248 leader_election.cc:288] T ef1e7d0183084efb9857bd039f690dcf P 77f4d396ff12475abe32edf4a915c40d [CANDIDATE]: Term 1 election: Requested vote from peers 5072f8b972474ce7bfd2d8da646f16d5 (127.0.206.67:35557), 1671860ad7cd49a5a9acfa9ebd4b8794 (127.0.206.65:45317)
I20260619 10:20:51.828722 18183 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ef1e7d0183084efb9857bd039f690dcf" candidate_uuid: "77f4d396ff12475abe32edf4a915c40d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5072f8b972474ce7bfd2d8da646f16d5"
I20260619 10:20:51.828708 17915 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ef1e7d0183084efb9857bd039f690dcf" candidate_uuid: "77f4d396ff12475abe32edf4a915c40d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1671860ad7cd49a5a9acfa9ebd4b8794"
I20260619 10:20:51.828795 18183 raft_consensus.cc:3037] T ef1e7d0183084efb9857bd039f690dcf P 5072f8b972474ce7bfd2d8da646f16d5 [term 0 FOLLOWER]: Advancing to term 1
I20260619 10:20:51.828795 17915 raft_consensus.cc:3037] T ef1e7d0183084efb9857bd039f690dcf P 1671860ad7cd49a5a9acfa9ebd4b8794 [term 0 FOLLOWER]: Advancing to term 1
I20260619 10:20:51.829630 17915 raft_consensus.cc:2454] T ef1e7d0183084efb9857bd039f690dcf P 1671860ad7cd49a5a9acfa9ebd4b8794 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 77f4d396ff12475abe32edf4a915c40d in term 1.
I20260619 10:20:51.829630 18183 raft_consensus.cc:2454] T ef1e7d0183084efb9857bd039f690dcf P 5072f8b972474ce7bfd2d8da646f16d5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 77f4d396ff12475abe32edf4a915c40d in term 1.
I20260619 10:20:51.829797 17982 leader_election.cc:302] T ef1e7d0183084efb9857bd039f690dcf P 77f4d396ff12475abe32edf4a915c40d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1671860ad7cd49a5a9acfa9ebd4b8794, 77f4d396ff12475abe32edf4a915c40d; no voters:
I20260619 10:20:51.829892 18248 raft_consensus.cc:2781] T ef1e7d0183084efb9857bd039f690dcf P 77f4d396ff12475abe32edf4a915c40d [term 1 FOLLOWER]: Leader election won for term 1
I20260619 10:20:51.830045 18248 raft_consensus.cc:686] T ef1e7d0183084efb9857bd039f690dcf P 77f4d396ff12475abe32edf4a915c40d [term 1 LEADER]: Becoming Leader. State: Replica: 77f4d396ff12475abe32edf4a915c40d, State: Running, Role: LEADER
I20260619 10:20:51.830137 18248 consensus_queue.cc:237] T ef1e7d0183084efb9857bd039f690dcf P 77f4d396ff12475abe32edf4a915c40d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5072f8b972474ce7bfd2d8da646f16d5" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 35557 } } peers { permanent_uuid: "77f4d396ff12475abe32edf4a915c40d" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 45237 } } peers { permanent_uuid: "1671860ad7cd49a5a9acfa9ebd4b8794" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 45317 } }
I20260619 10:20:51.830838 17769 catalog_manager.cc:5495] T ef1e7d0183084efb9857bd039f690dcf P 77f4d396ff12475abe32edf4a915c40d reported cstate change: term changed from 0 to 1, leader changed from <none> to 77f4d396ff12475abe32edf4a915c40d (127.0.206.66). New cstate: current_term: 1 leader_uuid: "77f4d396ff12475abe32edf4a915c40d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5072f8b972474ce7bfd2d8da646f16d5" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 35557 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "77f4d396ff12475abe32edf4a915c40d" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 45237 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1671860ad7cd49a5a9acfa9ebd4b8794" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 45317 } health_report { overall_health: UNKNOWN } } }
W20260619 10:20:51.841409 18096 tablet.cc:2369] T ef1e7d0183084efb9857bd039f690dcf P 77f4d396ff12475abe32edf4a915c40d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 10:20:51.844280 18242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:20:51.842270 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40506 (local address 127.0.206.66:45237)
0619 10:20:51.842300 (+ 30us) server_negotiation.cc:206] Beginning negotiation
0619 10:20:51.842302 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 10:20:51.842308 (+ 6us) server_negotiation.cc:407] Connection header received
0619 10:20:51.842330 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:20:51.842331 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:20:51.842345 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:20:51.842406 (+ 61us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 10:20:51.842503 (+ 97us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:20:51.843024 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:20:51.843708 (+ 684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:20:51.843876 (+ 168us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:20:51.843923 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:20:51.844039 (+ 116us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 10:20:51.844118 (+ 79us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 10:20:51.844181 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0619 10:20:51.844209 (+ 28us) server_negotiation.cc:299] Negotiation successful
0619 10:20:51.844222 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260619 10:20:51.845744 18183 raft_consensus.cc:1261] T ef1e7d0183084efb9857bd039f690dcf P 5072f8b972474ce7bfd2d8da646f16d5 [term 1 FOLLOWER]: Refusing update from remote peer 77f4d396ff12475abe32edf4a915c40d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 10:20:51.845762 17915 raft_consensus.cc:1261] T ef1e7d0183084efb9857bd039f690dcf P 1671860ad7cd49a5a9acfa9ebd4b8794 [term 1 FOLLOWER]: Refusing update from remote peer 77f4d396ff12475abe32edf4a915c40d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 10:20:51.846024 18248 consensus_queue.cc:1035] T ef1e7d0183084efb9857bd039f690dcf P 77f4d396ff12475abe32edf4a915c40d [LEADER]: Connected to new peer: Peer: permanent_uuid: "5072f8b972474ce7bfd2d8da646f16d5" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 35557 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 10:20:51.846139 18253 consensus_queue.cc:1035] T ef1e7d0183084efb9857bd039f690dcf P 77f4d396ff12475abe32edf4a915c40d [LEADER]: Connected to new peer: Peer: permanent_uuid: "1671860ad7cd49a5a9acfa9ebd4b8794" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 45317 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 10:20:51.850348 17770 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60398:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 10:20:51.850453 17770 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60398:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 10:20:51.851125 17770 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f3199a9fe74449d3aa7ca50e8a02ccd0: Sending DeleteTablet for 3 replicas of tablet ef1e7d0183084efb9857bd039f690dcf
I20260619 10:20:51.851492 18029 tablet_service.cc:1514] Processing DeleteTablet for tablet ef1e7d0183084efb9857bd039f690dcf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 10:20:51 UTC) from {username='kudu', principal='kudu/127.0.206.126@KRBTEST.COM'} at 127.0.0.1:40492
I20260619 10:20:51.851503 18163 tablet_service.cc:1514] Processing DeleteTablet for tablet ef1e7d0183084efb9857bd039f690dcf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 10:20:51 UTC) from {username='kudu', principal='kudu/127.0.206.126@KRBTEST.COM'} at 127.0.0.1:46706
I20260619 10:20:51.851805 825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTU9OP6/build/release/bin/kudu with pid 17832
I20260619 10:20:51.851835 17895 tablet_service.cc:1514] Processing DeleteTablet for tablet ef1e7d0183084efb9857bd039f690dcf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 10:20:51 UTC) from {username='kudu', principal='kudu/127.0.206.126@KRBTEST.COM'} at 127.0.0.1:60208
I20260619 10:20:51.853199 18267 tablet_replica.cc:330] T ef1e7d0183084efb9857bd039f690dcf P 5072f8b972474ce7bfd2d8da646f16d5: stopping tablet replica
I20260619 10:20:51.853300 18267 raft_consensus.cc:2229] T ef1e7d0183084efb9857bd039f690dcf P 5072f8b972474ce7bfd2d8da646f16d5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 10:20:51.853397 18267 raft_consensus.cc:2258] T ef1e7d0183084efb9857bd039f690dcf P 5072f8b972474ce7bfd2d8da646f16d5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 10:20:51.853535 18266 tablet_replica.cc:330] T ef1e7d0183084efb9857bd039f690dcf P 77f4d396ff12475abe32edf4a915c40d: stopping tablet replica
I20260619 10:20:51.853615 18266 raft_consensus.cc:2229] T ef1e7d0183084efb9857bd039f690dcf P 77f4d396ff12475abe32edf4a915c40d [term 1 LEADER]: Raft consensus shutting down.
I20260619 10:20:51.853694 18266 raft_consensus.cc:2258] T ef1e7d0183084efb9857bd039f690dcf P 77f4d396ff12475abe32edf4a915c40d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 10:20:51.854103 18266 ts_tablet_manager.cc:1905] T ef1e7d0183084efb9857bd039f690dcf P 77f4d396ff12475abe32edf4a915c40d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 10:20:51.854171 18267 ts_tablet_manager.cc:1905] T ef1e7d0183084efb9857bd039f690dcf P 5072f8b972474ce7bfd2d8da646f16d5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 10:20:51.855993 18267 ts_tablet_manager.cc:1918] T ef1e7d0183084efb9857bd039f690dcf P 5072f8b972474ce7bfd2d8da646f16d5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 10:20:51.856099 18266 ts_tablet_manager.cc:1918] T ef1e7d0183084efb9857bd039f690dcf P 77f4d396ff12475abe32edf4a915c40d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 10:20:51.856119 18267 log.cc:1196] T ef1e7d0183084efb9857bd039f690dcf P 5072f8b972474ce7bfd2d8da646f16d5: Deleting WAL directory at /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781864224577058-825-0/minicluster-data/ts-2/wal/wals/ef1e7d0183084efb9857bd039f690dcf
I20260619 10:20:51.856244 18266 log.cc:1196] T ef1e7d0183084efb9857bd039f690dcf P 77f4d396ff12475abe32edf4a915c40d: Deleting WAL directory at /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781864224577058-825-0/minicluster-data/ts-1/wal/wals/ef1e7d0183084efb9857bd039f690dcf
I20260619 10:20:51.856400 18267 ts_tablet_manager.cc:1939] T ef1e7d0183084efb9857bd039f690dcf P 5072f8b972474ce7bfd2d8da646f16d5: Deleting consensus metadata
I20260619 10:20:51.856581 18266 ts_tablet_manager.cc:1939] T ef1e7d0183084efb9857bd039f690dcf P 77f4d396ff12475abe32edf4a915c40d: Deleting consensus metadata
I20260619 10:20:51.856829 17757 catalog_manager.cc:4841] TS 5072f8b972474ce7bfd2d8da646f16d5 (127.0.206.67:35557): tablet ef1e7d0183084efb9857bd039f690dcf (table test-table [id=f3f0d13da0814e3fb07610a6d680e004]) successfully deleted
I20260619 10:20:51.857023 17755 catalog_manager.cc:4841] TS 77f4d396ff12475abe32edf4a915c40d (127.0.206.66:45237): tablet ef1e7d0183084efb9857bd039f690dcf (table test-table [id=f3f0d13da0814e3fb07610a6d680e004]) successfully deleted
W20260619 10:20:51.858232 17754 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.0.206.65:45317 (error 108)
I20260619 10:20:51.858439 825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTU9OP6/build/release/bin/kudu with pid 17966
I20260619 10:20:51.858990 18239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:20:51.858540 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.65:45317 (local address 127.0.0.1:60216)
0619 10:20:51.858917 (+ 377us) negotiation.cc:107] Waiting for socket to connect
0619 10:20:51.858953 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.65:45317: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":350}
W20260619 10:20:51.859257 17754 catalog_manager.cc:4584] TS 1671860ad7cd49a5a9acfa9ebd4b8794 (127.0.206.65:45317): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet ef1e7d0183084efb9857bd039f690dcf: Network error: Client connection negotiation failed: client connection to 127.0.206.65:45317: connect: Connection refused (error 111)
I20260619 10:20:51.863332 825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTU9OP6/build/release/bin/kudu with pid 18100
I20260619 10:20:51.868185 825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTU9OP6/build/release/bin/kudu with pid 17738
2026-06-19T10:20:51Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (1851 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-06-19T10:20:51Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T10:20:51Z Disabled control of system clock
I20260619 10:20:51.894351 825 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTU9OP6/build/release/bin/kudu
/tmp/dist-test-taskTU9OP6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781864224577058-825-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781864224577058-825-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781864224577058-825-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781864224577058-825-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.206.126:38687
--webserver_interface=127.0.206.126
--webserver_port=0
--builtin_ntp_servers=127.0.206.84:38561
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.206.126:38687
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260619 10:20:51.972357 18272 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 10:20:51.972554 18272 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 10:20:51.972572 18272 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 10:20:51.973598 18272 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 10:20:51.973631 18272 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 10:20:51.973644 18272 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 10:20:51.973656 18272 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 10:20:51.973668 18272 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 10:20:51.974936 18272 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.206.84:38561
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781864224577058-825-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781864224577058-825-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.206.126:38687
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.206.126:38687
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781864224577058-825-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.206.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18272
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781864224577058-825-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 09:32:44 UTC on ccycloud.quasar-gbtldp.root.comops.site
build id 4455440
I20260619 10:20:51.975122 18272 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 10:20:51.975338 18272 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 10:20:51.979885 18280 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 10:20:51.979848 18277 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 10:20:51.979864 18278 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 10:20:51.979907 18272 server_base.cc:1034] running on GCE node
I20260619 10:20:51.980278 18272 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 10:20:51.980589 18272 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 10:20:51.981750 18272 hybrid_clock.cc:648] HybridClock initialized: now 1781864451981746 us; error 35 us; skew 500 ppm
I20260619 10:20:51.982911 18272 webserver.cc:460] Webserver started at http://127.0.206.126:46235/ using document root <none> and password file <none>
I20260619 10:20:51.983162 18272 fs_manager.cc:359] Metadata directory not provided
I20260619 10:20:51.983228 18272 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 10:20:51.983369 18272 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 10:20:51.984362 18272 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781864224577058-825-0/minicluster-data/master-0/data/instance:
uuid: "93286777d79d4689858b23f69f34c428"
format_stamp: "Formatted at 2026-06-19 10:20:51 on dist-test-slave-6pt2"
server_key: "f17dc9f1f33e879d6217e309bc69db81"
server_key_iv: "cfe9e9f8644f4732574fa5f836dd4131"
server_key_version: "encryptionkey@0"
I20260619 10:20:51.984695 18272 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781864224577058-825-0/minicluster-data/master-0/wal/instance:
uuid: "93286777d79d4689858b23f69f34c428"
format_stamp: "Formatted at 2026-06-19 10:20:51 on dist-test-slave-6pt2"
server_key: "f17dc9f1f33e879d6217e309bc69db81"
server_key_iv: "cfe9e9f8644f4732574fa5f836dd4131"
server_key_version: "encryptionkey@0"
I20260619 10:20:51.986129 18272 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260619 10:20:51.986860 18286 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 10:20:51.987032 18272 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260619 10:20:51.987130 18272 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781864224577058-825-0/minicluster-data/master-0/data,/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781864224577058-825-0/minicluster-data/master-0/wal
uuid: "93286777d79d4689858b23f69f34c428"
format_stamp: "Formatted at 2026-06-19 10:20:51 on dist-test-slave-6pt2"
server_key: "f17dc9f1f33e879d6217e309bc69db81"
server_key_iv: "cfe9e9f8644f4732574fa5f836dd4131"
server_key_version: "encryptionkey@0"
I20260619 10:20:51.987212 18272 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781864224577058-825-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781864224577058-825-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781864224577058-825-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 10:20:51.995163 18272 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 10:20:51.995414 18272 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 10:20:51.995545 18272 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 10:20:51.999637 18272 rpc_server.cc:307] RPC server started. Bound to: 127.0.206.126:38687
I20260619 10:20:51.999670 18338 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.206.126:38687 every 8 connection(s)
I20260619 10:20:51.999985 18272 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781864224577058-825-0/minicluster-data/master-0/data/info.pb
I20260619 10:20:52.000936 18339 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 10:20:52.003130 18339 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 93286777d79d4689858b23f69f34c428: Bootstrap starting.
I20260619 10:20:52.003675 18339 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 93286777d79d4689858b23f69f34c428: Neither blocks nor log segments found. Creating new log.
I20260619 10:20:52.003969 18339 log.cc:824] T 00000000000000000000000000000000 P 93286777d79d4689858b23f69f34c428: Log is configured to *not* fsync() on all Append() calls
I20260619 10:20:52.004583 18339 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 93286777d79d4689858b23f69f34c428: No bootstrap required, opened a new log
I20260619 10:20:52.006127 18339 raft_consensus.cc:348] T 00000000000000000000000000000000 P 93286777d79d4689858b23f69f34c428 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "93286777d79d4689858b23f69f34c428" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 38687 } }
I20260619 10:20:52.006254 18339 raft_consensus.cc:374] T 00000000000000000000000000000000 P 93286777d79d4689858b23f69f34c428 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 10:20:52.006316 18339 raft_consensus.cc:729] T 00000000000000000000000000000000 P 93286777d79d4689858b23f69f34c428 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 93286777d79d4689858b23f69f34c428, State: Initialized, Role: FOLLOWER
I20260619 10:20:52.006459 18339 consensus_queue.cc:260] T 00000000000000000000000000000000 P 93286777d79d4689858b23f69f34c428 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "93286777d79d4689858b23f69f34c428" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 38687 } }
I20260619 10:20:52.006520 18339 raft_consensus.cc:388] T 00000000000000000000000000000000 P 93286777d79d4689858b23f69f34c428 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 10:20:52.006541 18339 raft_consensus.cc:482] T 00000000000000000000000000000000 P 93286777d79d4689858b23f69f34c428 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 10:20:52.006639 18339 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 93286777d79d4689858b23f69f34c428 [term 0 FOLLOWER]: Advancing to term 1
I20260619 10:20:52.007176 18339 raft_consensus.cc:504] T 00000000000000000000000000000000 P 93286777d79d4689858b23f69f34c428 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "93286777d79d4689858b23f69f34c428" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 38687 } }
I20260619 10:20:52.007292 18339 leader_election.cc:302] T 00000000000000000000000000000000 P 93286777d79d4689858b23f69f34c428 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 93286777d79d4689858b23f69f34c428; no voters:
I20260619 10:20:52.007470 18339 leader_election.cc:288] T 00000000000000000000000000000000 P 93286777d79d4689858b23f69f34c428 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 10:20:52.007493 18342 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 93286777d79d4689858b23f69f34c428 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 10:20:52.007652 18342 raft_consensus.cc:686] T 00000000000000000000000000000000 P 93286777d79d4689858b23f69f34c428 [term 1 LEADER]: Becoming Leader. State: Replica: 93286777d79d4689858b23f69f34c428, State: Running, Role: LEADER
I20260619 10:20:52.007723 18339 sys_catalog.cc:564] T 00000000000000000000000000000000 P 93286777d79d4689858b23f69f34c428 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 10:20:52.007782 18342 consensus_queue.cc:237] T 00000000000000000000000000000000 P 93286777d79d4689858b23f69f34c428 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "93286777d79d4689858b23f69f34c428" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 38687 } }
I20260619 10:20:52.008131 18344 sys_catalog.cc:455] T 00000000000000000000000000000000 P 93286777d79d4689858b23f69f34c428 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 93286777d79d4689858b23f69f34c428. Latest consensus state: current_term: 1 leader_uuid: "93286777d79d4689858b23f69f34c428" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "93286777d79d4689858b23f69f34c428" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 38687 } } }
I20260619 10:20:52.008265 825 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTU9OP6/build/release/bin/kudu as pid 18272
I20260619 10:20:52.008289 18344 sys_catalog.cc:458] T 00000000000000000000000000000000 P 93286777d79d4689858b23f69f34c428 [sys.catalog]: This master's current role is: LEADER
I20260619 10:20:52.008371 825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781864224577058-825-0/minicluster-data/master-0/wal/instance
I20260619 10:20:52.008205 18343 sys_catalog.cc:455] T 00000000000000000000000000000000 P 93286777d79d4689858b23f69f34c428 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "93286777d79d4689858b23f69f34c428" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "93286777d79d4689858b23f69f34c428" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 38687 } } }
I20260619 10:20:52.008460 18343 sys_catalog.cc:458] T 00000000000000000000000000000000 P 93286777d79d4689858b23f69f34c428 [sys.catalog]: This master's current role is: LEADER
I20260619 10:20:52.008555 18351 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 10:20:52.008585 825 external_mini_cluster.cc:1428] Setting key db57e3dbd914adb7483dc9239643f1ab
I20260619 10:20:52.009227 18351 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 10:20:52.010843 18351 catalog_manager.cc:1347] Generated new cluster ID: 40e79b03c25c4cc988a04792d21cf9e2
I20260619 10:20:52.010892 18351 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 10:20:52.015228 18360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:20:52.009131 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37250 (local address 127.0.206.126:38687)
0619 10:20:52.009640 (+ 509us) server_negotiation.cc:206] Beginning negotiation
0619 10:20:52.009648 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 10:20:52.009678 (+ 30us) server_negotiation.cc:407] Connection header received
0619 10:20:52.009925 (+ 247us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:20:52.009932 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:20:52.010002 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:20:52.010148 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:20:52.010334 (+ 186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:20:52.011644 (+ 1310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:20:52.012800 (+ 1156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:20:52.013411 (+ 611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:20:52.013524 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:20:52.014418 (+ 894us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 10:20:52.014421 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 10:20:52.014422 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 10:20:52.014630 (+ 208us) server_negotiation.cc:922] calling sasl_server_start()
0619 10:20:52.014672 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 10:20:52.014686 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 10:20:52.014746 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0619 10:20:52.014947 (+ 201us) server_negotiation.cc:299] Negotiation successful
0619 10:20:52.015022 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":444,"thread_start_us":57,"threads_started":1}
I20260619 10:20:52.015588 18351 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 10:20:52.016047 18351 catalog_manager.cc:1504] Loading token signing keys...
I20260619 10:20:52.018648 18351 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 93286777d79d4689858b23f69f34c428: Generated new TSK 0
I20260619 10:20:52.018821 18351 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 10:20:52.021991 825 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTU9OP6/build/release/bin/kudu
/tmp/dist-test-taskTU9OP6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781864224577058-825-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781864224577058-825-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781864224577058-825-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781864224577058-825-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.206.65:0
--local_ip_for_outbound_sockets=127.0.206.65
--webserver_interface=127.0.206.65
--webserver_port=0
--tserver_master_addrs=127.0.206.126:38687
--builtin_ntp_servers=127.0.206.84:38561
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260619 10:20:52.101004 18363 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 10:20:52.101195 18363 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 10:20:52.101217 18363 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 10:20:52.102360 18363 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 10:20:52.102406 18363 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 10:20:52.102432 18363 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.206.65
I20260619 10:20:52.103807 18363 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.206.84:38561
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781864224577058-825-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781864224577058-825-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.206.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781864224577058-825-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.206.65
--webserver_port=0
--tserver_master_addrs=127.0.206.126:38687
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18363
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.206.65
--log_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781864224577058-825-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 09:32:44 UTC on ccycloud.quasar-gbtldp.root.comops.site
build id 4455440
I20260619 10:20:52.104029 18363 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 10:20:52.104349 18363 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 10:20:52.110113 18369 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 10:20:52.110162 18371 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 10:20:52.110113 18368 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 10:20:52.110384 18363 server_base.cc:1034] running on GCE node
I20260619 10:20:52.110672 18363 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 10:20:52.110925 18363 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 10:20:52.112083 18363 hybrid_clock.cc:648] HybridClock initialized: now 1781864452112073 us; error 30 us; skew 500 ppm
I20260619 10:20:52.113329 18363 webserver.cc:460] Webserver started at http://127.0.206.65:37177/ using document root <none> and password file <none>
I20260619 10:20:52.113564 18363 fs_manager.cc:359] Metadata directory not provided
I20260619 10:20:52.113629 18363 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 10:20:52.113758 18363 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 10:20:52.114861 18363 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781864224577058-825-0/minicluster-data/ts-0/data/instance:
uuid: "1b88d465d65c4954870bc4c8de4c74cb"
format_stamp: "Formatted at 2026-06-19 10:20:52 on dist-test-slave-6pt2"
server_key: "c2c4b62010e75ea4c43a1933b19c205a"
server_key_iv: "6a6abf20fd85c961e52021c4ce877d13"
server_key_version: "encryptionkey@0"
I20260619 10:20:52.115221 18363 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781864224577058-825-0/minicluster-data/ts-0/wal/instance:
uuid: "1b88d465d65c4954870bc4c8de4c74cb"
format_stamp: "Formatted at 2026-06-19 10:20:52 on dist-test-slave-6pt2"
server_key: "c2c4b62010e75ea4c43a1933b19c205a"
server_key_iv: "6a6abf20fd85c961e52021c4ce877d13"
server_key_version: "encryptionkey@0"
I20260619 10:20:52.116559 18363 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 10:20:52.117300 18377 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 10:20:52.117473 18363 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 10:20:52.117570 18363 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781864224577058-825-0/minicluster-data/ts-0/data,/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781864224577058-825-0/minicluster-data/ts-0/wal
uuid: "1b88d465d65c4954870bc4c8de4c74cb"
format_stamp: "Formatted at 2026-06-19 10:20:52 on dist-test-slave-6pt2"
server_key: "c2c4b62010e75ea4c43a1933b19c205a"
server_key_iv: "6a6abf20fd85c961e52021c4ce877d13"
server_key_version: "encryptionkey@0"
I20260619 10:20:52.117650 18363 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781864224577058-825-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781864224577058-825-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781864224577058-825-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 10:20:52.132004 18363 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 10:20:52.132305 18363 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 10:20:52.132455 18363 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 10:20:52.132674 18363 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 10:20:52.132993 18363 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 10:20:52.133067 18363 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 10:20:52.133107 18363 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 10:20:52.133136 18363 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 10:20:52.139006 18363 rpc_server.cc:307] RPC server started. Bound to: 127.0.206.65:42695
I20260619 10:20:52.139098 18490 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.206.65:42695 every 8 connection(s)
I20260619 10:20:52.139405 18363 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781864224577058-825-0/minicluster-data/ts-0/data/info.pb
I20260619 10:20:52.144479 18360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:20:52.140121 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.65:50619 (local address 127.0.206.126:38687)
0619 10:20:52.140184 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0619 10:20:52.140186 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 10:20:52.140413 (+ 227us) server_negotiation.cc:407] Connection header received
0619 10:20:52.140919 (+ 506us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:20:52.140920 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:20:52.140933 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:20:52.140979 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:20:52.141626 (+ 647us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:20:52.142104 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:20:52.143079 (+ 975us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:20:52.143283 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:20:52.143363 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:20:52.144174 (+ 811us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 10:20:52.144176 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 10:20:52.144176 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 10:20:52.144193 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0619 10:20:52.144206 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 10:20:52.144211 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 10:20:52.144256 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0619 10:20:52.144346 (+ 90us) server_negotiation.cc:299] Negotiation successful
0619 10:20:52.144385 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260619 10:20:52.144681 18493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:20:52.140131 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:38687 (local address 127.0.206.65:50619)
0619 10:20:52.140290 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0619 10:20:52.140321 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0619 10:20:52.140815 (+ 494us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 10:20:52.141044 (+ 229us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 10:20:52.141050 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 10:20:52.141088 (+ 38us) client_negotiation.cc:189] Negotiated authn=SASL
0619 10:20:52.141535 (+ 447us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:20:52.141539 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:20:52.142195 (+ 656us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:20:52.142196 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:20:52.142958 (+ 762us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:20:52.142960 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:20:52.143372 (+ 412us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:20:52.143373 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:20:52.143501 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:20:52.144034 (+ 533us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 10:20:52.144034 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0619 10:20:52.144060 (+ 26us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 10:20:52.144081 (+ 21us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 10:20:52.144256 (+ 175us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 10:20:52.144261 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 10:20:52.144264 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 10:20:52.144349 (+ 85us) client_negotiation.cc:240] Negotiation successful
0619 10:20:52.144468 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":64,"threads_started":1}
I20260619 10:20:52.145159 18491 heartbeater.cc:347] Connected to a master server at 127.0.206.126:38687
I20260619 10:20:52.145238 18491 heartbeater.cc:464] Registering TS with master...
I20260619 10:20:52.145418 18491 heartbeater.cc:511] Master 127.0.206.126:38687 requested a full tablet report, sending...
I20260619 10:20:52.145884 18303 ts_manager.cc:194] Registered new tserver with Master: 1b88d465d65c4954870bc4c8de4c74cb (127.0.206.65:42695)
I20260619 10:20:52.146466 18303 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.206.65:50619
I20260619 10:20:52.146518 825 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTU9OP6/build/release/bin/kudu as pid 18363
I20260619 10:20:52.146641 825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781864224577058-825-0/minicluster-data/ts-0/wal/instance
I20260619 10:20:52.146886 825 external_mini_cluster.cc:1428] Setting key e8ee9c0a3acd748eee1033199bb60a70
I20260619 10:20:52.147670 825 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTU9OP6/build/release/bin/kudu
/tmp/dist-test-taskTU9OP6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781864224577058-825-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781864224577058-825-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781864224577058-825-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781864224577058-825-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.206.66:0
--local_ip_for_outbound_sockets=127.0.206.66
--webserver_interface=127.0.206.66
--webserver_port=0
--tserver_master_addrs=127.0.206.126:38687
--builtin_ntp_servers=127.0.206.84:38561
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260619 10:20:52.225917 18494 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 10:20:52.226096 18494 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 10:20:52.226116 18494 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 10:20:52.227156 18494 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 10:20:52.227190 18494 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 10:20:52.227213 18494 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.206.66
I20260619 10:20:52.228363 18494 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.206.84:38561
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781864224577058-825-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781864224577058-825-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.206.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781864224577058-825-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.206.66
--webserver_port=0
--tserver_master_addrs=127.0.206.126:38687
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18494
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.206.66
--log_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781864224577058-825-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 09:32:44 UTC on ccycloud.quasar-gbtldp.root.comops.site
build id 4455440
I20260619 10:20:52.228545 18494 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 10:20:52.228750 18494 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260619 10:20:52.233366 18494 server_base.cc:1034] running on GCE node
W20260619 10:20:52.233328 18499 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 10:20:52.233335 18502 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 10:20:52.233373 18500 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 10:20:52.233762 18494 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 10:20:52.234009 18494 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 10:20:52.235163 18494 hybrid_clock.cc:648] HybridClock initialized: now 1781864452235157 us; error 34 us; skew 500 ppm
I20260619 10:20:52.236346 18494 webserver.cc:460] Webserver started at http://127.0.206.66:43239/ using document root <none> and password file <none>
I20260619 10:20:52.236572 18494 fs_manager.cc:359] Metadata directory not provided
I20260619 10:20:52.236639 18494 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 10:20:52.236766 18494 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 10:20:52.237751 18494 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781864224577058-825-0/minicluster-data/ts-1/data/instance:
uuid: "cb6d28bca45f4cb2b406e09691fa9f69"
format_stamp: "Formatted at 2026-06-19 10:20:52 on dist-test-slave-6pt2"
server_key: "5f0bd1a868e058bac5bcb4850b6b9323"
server_key_iv: "837d008c27e8c5246e77d470626348a6"
server_key_version: "encryptionkey@0"
I20260619 10:20:52.238095 18494 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781864224577058-825-0/minicluster-data/ts-1/wal/instance:
uuid: "cb6d28bca45f4cb2b406e09691fa9f69"
format_stamp: "Formatted at 2026-06-19 10:20:52 on dist-test-slave-6pt2"
server_key: "5f0bd1a868e058bac5bcb4850b6b9323"
server_key_iv: "837d008c27e8c5246e77d470626348a6"
server_key_version: "encryptionkey@0"
I20260619 10:20:52.239471 18494 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260619 10:20:52.240163 18508 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 10:20:52.240319 18494 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260619 10:20:52.240414 18494 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781864224577058-825-0/minicluster-data/ts-1/data,/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781864224577058-825-0/minicluster-data/ts-1/wal
uuid: "cb6d28bca45f4cb2b406e09691fa9f69"
format_stamp: "Formatted at 2026-06-19 10:20:52 on dist-test-slave-6pt2"
server_key: "5f0bd1a868e058bac5bcb4850b6b9323"
server_key_iv: "837d008c27e8c5246e77d470626348a6"
server_key_version: "encryptionkey@0"
I20260619 10:20:52.240491 18494 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781864224577058-825-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781864224577058-825-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781864224577058-825-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 10:20:52.253358 18494 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 10:20:52.253620 18494 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 10:20:52.253759 18494 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 10:20:52.253959 18494 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 10:20:52.254246 18494 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 10:20:52.254298 18494 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 10:20:52.254335 18494 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 10:20:52.254362 18494 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 10:20:52.260103 18494 rpc_server.cc:307] RPC server started. Bound to: 127.0.206.66:36347
I20260619 10:20:52.260142 18621 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.206.66:36347 every 8 connection(s)
I20260619 10:20:52.260432 18494 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781864224577058-825-0/minicluster-data/ts-1/data/info.pb
I20260619 10:20:52.261891 825 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTU9OP6/build/release/bin/kudu as pid 18494
I20260619 10:20:52.262001 825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781864224577058-825-0/minicluster-data/ts-1/wal/instance
I20260619 10:20:52.262197 825 external_mini_cluster.cc:1428] Setting key 7521fb8242ca7290ef969eaf2141b909
I20260619 10:20:52.263164 825 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTU9OP6/build/release/bin/kudu
/tmp/dist-test-taskTU9OP6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781864224577058-825-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781864224577058-825-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781864224577058-825-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781864224577058-825-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.206.67:0
--local_ip_for_outbound_sockets=127.0.206.67
--webserver_interface=127.0.206.67
--webserver_port=0
--tserver_master_addrs=127.0.206.126:38687
--builtin_ntp_servers=127.0.206.84:38561
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260619 10:20:52.265676 18360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:20:52.261047 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.66:57207 (local address 127.0.206.126:38687)
0619 10:20:52.261116 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0619 10:20:52.261118 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 10:20:52.261340 (+ 222us) server_negotiation.cc:407] Connection header received
0619 10:20:52.261791 (+ 451us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:20:52.261792 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:20:52.261805 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:20:52.261885 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:20:52.262687 (+ 802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:20:52.263181 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:20:52.264152 (+ 971us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:20:52.264475 (+ 323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:20:52.264548 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:20:52.265363 (+ 815us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 10:20:52.265365 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 10:20:52.265365 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 10:20:52.265370 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0619 10:20:52.265386 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 10:20:52.265390 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 10:20:52.265456 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0619 10:20:52.265526 (+ 70us) server_negotiation.cc:299] Negotiation successful
0619 10:20:52.265561 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260619 10:20:52.265798 18624 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:20:52.261018 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:38687 (local address 127.0.206.66:57207)
0619 10:20:52.261211 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0619 10:20:52.261246 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0619 10:20:52.261674 (+ 428us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 10:20:52.261958 (+ 284us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 10:20:52.261961 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 10:20:52.261968 (+ 7us) client_negotiation.cc:189] Negotiated authn=SASL
0619 10:20:52.262566 (+ 598us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:20:52.262573 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:20:52.263279 (+ 706us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:20:52.263280 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:20:52.264047 (+ 767us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:20:52.264050 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:20:52.264562 (+ 512us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:20:52.264563 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:20:52.264684 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:20:52.265188 (+ 504us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 10:20:52.265188 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0619 10:20:52.265215 (+ 27us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 10:20:52.265235 (+ 20us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 10:20:52.265457 (+ 222us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 10:20:52.265460 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 10:20:52.265462 (+ 2us) client_negotiation.cc:769] Sending connection context
0619 10:20:52.265531 (+ 69us) client_negotiation.cc:240] Negotiation successful
0619 10:20:52.265607 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":109,"threads_started":1}
I20260619 10:20:52.266271 18622 heartbeater.cc:347] Connected to a master server at 127.0.206.126:38687
I20260619 10:20:52.266350 18622 heartbeater.cc:464] Registering TS with master...
I20260619 10:20:52.266568 18622 heartbeater.cc:511] Master 127.0.206.126:38687 requested a full tablet report, sending...
I20260619 10:20:52.266974 18303 ts_manager.cc:194] Registered new tserver with Master: cb6d28bca45f4cb2b406e09691fa9f69 (127.0.206.66:36347)
I20260619 10:20:52.267509 18303 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.206.66:57207
W20260619 10:20:52.342778 18625 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 10:20:52.342998 18625 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 10:20:52.343020 18625 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 10:20:52.344192 18625 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 10:20:52.344229 18625 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 10:20:52.344254 18625 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.206.67
I20260619 10:20:52.345506 18625 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.206.84:38561
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781864224577058-825-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781864224577058-825-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.206.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781864224577058-825-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.206.67
--webserver_port=0
--tserver_master_addrs=127.0.206.126:38687
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18625
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.206.67
--log_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781864224577058-825-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 09:32:44 UTC on ccycloud.quasar-gbtldp.root.comops.site
build id 4455440
I20260619 10:20:52.345675 18625 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 10:20:52.345880 18625 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 10:20:52.350323 18631 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 10:20:52.350324 18630 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 10:20:52.350436 18625 server_base.cc:1034] running on GCE node
W20260619 10:20:52.350360 18633 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 10:20:52.350800 18625 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 10:20:52.351019 18625 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 10:20:52.352170 18625 hybrid_clock.cc:648] HybridClock initialized: now 1781864452352159 us; error 27 us; skew 500 ppm
I20260619 10:20:52.353348 18625 webserver.cc:460] Webserver started at http://127.0.206.67:46595/ using document root <none> and password file <none>
I20260619 10:20:52.353528 18625 fs_manager.cc:359] Metadata directory not provided
I20260619 10:20:52.353565 18625 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 10:20:52.353649 18625 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 10:20:52.354565 18625 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781864224577058-825-0/minicluster-data/ts-2/data/instance:
uuid: "c0f80e4da8354151a97f1bcfb156bfa8"
format_stamp: "Formatted at 2026-06-19 10:20:52 on dist-test-slave-6pt2"
server_key: "ff519fd43f6a1876c9d28aaf88c8dd04"
server_key_iv: "196bbc5bbe1c4160f531989487907404"
server_key_version: "encryptionkey@0"
I20260619 10:20:52.354915 18625 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781864224577058-825-0/minicluster-data/ts-2/wal/instance:
uuid: "c0f80e4da8354151a97f1bcfb156bfa8"
format_stamp: "Formatted at 2026-06-19 10:20:52 on dist-test-slave-6pt2"
server_key: "ff519fd43f6a1876c9d28aaf88c8dd04"
server_key_iv: "196bbc5bbe1c4160f531989487907404"
server_key_version: "encryptionkey@0"
I20260619 10:20:52.356091 18625 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 10:20:52.356776 18639 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 10:20:52.357172 18625 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 10:20:52.357281 18625 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781864224577058-825-0/minicluster-data/ts-2/data,/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781864224577058-825-0/minicluster-data/ts-2/wal
uuid: "c0f80e4da8354151a97f1bcfb156bfa8"
format_stamp: "Formatted at 2026-06-19 10:20:52 on dist-test-slave-6pt2"
server_key: "ff519fd43f6a1876c9d28aaf88c8dd04"
server_key_iv: "196bbc5bbe1c4160f531989487907404"
server_key_version: "encryptionkey@0"
I20260619 10:20:52.357362 18625 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781864224577058-825-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781864224577058-825-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781864224577058-825-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 10:20:52.369875 18625 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 10:20:52.370121 18625 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 10:20:52.370260 18625 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 10:20:52.370481 18625 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 10:20:52.370801 18625 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 10:20:52.370851 18625 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 10:20:52.370908 18625 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 10:20:52.370936 18625 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 10:20:52.376152 18625 rpc_server.cc:307] RPC server started. Bound to: 127.0.206.67:34987
I20260619 10:20:52.376212 18752 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.206.67:34987 every 8 connection(s)
I20260619 10:20:52.376516 18625 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781864224577058-825-0/minicluster-data/ts-2/data/info.pb
I20260619 10:20:52.378021 825 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTU9OP6/build/release/bin/kudu as pid 18625
I20260619 10:20:52.378129 825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781864224577058-825-0/minicluster-data/ts-2/wal/instance
I20260619 10:20:52.378316 825 external_mini_cluster.cc:1428] Setting key d57bb5fe1540325ce3f8a085a2e2f72e
I20260619 10:20:52.381608 18360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:20:52.377162 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.67:54271 (local address 127.0.206.126:38687)
0619 10:20:52.377206 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0619 10:20:52.377208 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 10:20:52.377431 (+ 223us) server_negotiation.cc:407] Connection header received
0619 10:20:52.377928 (+ 497us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:20:52.377929 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:20:52.377942 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:20:52.377988 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:20:52.378841 (+ 853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:20:52.379358 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:20:52.380344 (+ 986us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:20:52.380553 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:20:52.380639 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:20:52.381353 (+ 714us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 10:20:52.381354 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 10:20:52.381355 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 10:20:52.381359 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0619 10:20:52.381371 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 10:20:52.381375 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 10:20:52.381426 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0619 10:20:52.381507 (+ 81us) server_negotiation.cc:299] Negotiation successful
0619 10:20:52.381534 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260619 10:20:52.381809 18755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:20:52.377143 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:38687 (local address 127.0.206.67:54271)
0619 10:20:52.377324 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0619 10:20:52.377354 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0619 10:20:52.377818 (+ 464us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 10:20:52.378062 (+ 244us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 10:20:52.378066 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 10:20:52.378075 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0619 10:20:52.378746 (+ 671us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:20:52.378751 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:20:52.379462 (+ 711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:20:52.379463 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:20:52.380233 (+ 770us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:20:52.380235 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:20:52.380648 (+ 413us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:20:52.380649 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:20:52.380782 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:20:52.381212 (+ 430us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 10:20:52.381212 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0619 10:20:52.381236 (+ 24us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 10:20:52.381257 (+ 21us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 10:20:52.381444 (+ 187us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 10:20:52.381448 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 10:20:52.381450 (+ 2us) client_negotiation.cc:769] Sending connection context
0619 10:20:52.381511 (+ 61us) client_negotiation.cc:240] Negotiation successful
0619 10:20:52.381600 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":74,"threads_started":1}
I20260619 10:20:52.382220 18753 heartbeater.cc:347] Connected to a master server at 127.0.206.126:38687
I20260619 10:20:52.382300 18753 heartbeater.cc:464] Registering TS with master...
I20260619 10:20:52.382436 18753 heartbeater.cc:511] Master 127.0.206.126:38687 requested a full tablet report, sending...
I20260619 10:20:52.382779 18303 ts_manager.cc:194] Registered new tserver with Master: c0f80e4da8354151a97f1bcfb156bfa8 (127.0.206.67:34987)
I20260619 10:20:52.383090 18303 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.206.67:54271
I20260619 10:20:52.389971 825 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 10:20:52.394105 18360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:20:52.391379 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37254 (local address 127.0.206.126:38687)
0619 10:20:52.391461 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0619 10:20:52.391463 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 10:20:52.391558 (+ 95us) server_negotiation.cc:407] Connection header received
0619 10:20:52.391625 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:20:52.391626 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:20:52.391638 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:20:52.391708 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:20:52.391877 (+ 169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:20:52.392329 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:20:52.393068 (+ 739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:20:52.393255 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:20:52.393339 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:20:52.393888 (+ 549us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 10:20:52.393889 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 10:20:52.393889 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 10:20:52.393894 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0619 10:20:52.393905 (+ 11us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 10:20:52.393909 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 10:20:52.393956 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0619 10:20:52.394005 (+ 49us) server_negotiation.cc:299] Negotiation successful
0619 10:20:52.394026 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260619 10:20:52.395476 18303 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37254:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 10:20:52.396900 18303 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 10:20:52.402901 18764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:20:52.399043 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42830 (local address 127.0.206.67:34987)
0619 10:20:52.399546 (+ 503us) server_negotiation.cc:206] Beginning negotiation
0619 10:20:52.399550 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 10:20:52.399566 (+ 16us) server_negotiation.cc:407] Connection header received
0619 10:20:52.399619 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:20:52.399621 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:20:52.399635 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:20:52.399713 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 10:20:52.399939 (+ 226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:20:52.400626 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:20:52.402003 (+ 1377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:20:52.402506 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:20:52.402595 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:20:52.402699 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0619 10:20:52.402792 (+ 93us) server_negotiation.cc:299] Negotiation successful
0619 10:20:52.402825 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":473,"thread_start_us":82,"threads_started":1}
I20260619 10:20:52.402900 18761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:20:52.398885 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.67:34987 (local address 127.0.0.1:42830)
0619 10:20:52.398996 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0619 10:20:52.399031 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0619 10:20:52.399096 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 10:20:52.399725 (+ 629us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 10:20:52.399726 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 10:20:52.399728 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 10:20:52.399777 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:20:52.399780 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:20:52.400713 (+ 933us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:20:52.400714 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:20:52.401911 (+ 1197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:20:52.401914 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:20:52.402595 (+ 681us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:20:52.402596 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:20:52.402693 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:20:52.402706 (+ 13us) client_negotiation.cc:769] Sending connection context
0619 10:20:52.402779 (+ 73us) client_negotiation.cc:240] Negotiation successful
0619 10:20:52.402826 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":91,"thread_start_us":71,"threads_started":1}
I20260619 10:20:52.403162 18763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:20:52.399026 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.65:42695 (local address 127.0.0.1:40166)
0619 10:20:52.399370 (+ 344us) negotiation.cc:107] Waiting for socket to connect
0619 10:20:52.399381 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 10:20:52.399441 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 10:20:52.399686 (+ 245us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 10:20:52.399691 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 10:20:52.399694 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 10:20:52.399752 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:20:52.399755 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:20:52.401012 (+ 1257us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:20:52.401013 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:20:52.402039 (+ 1026us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:20:52.402041 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:20:52.402962 (+ 921us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:20:52.402963 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:20:52.403022 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:20:52.403025 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 10:20:52.403069 (+ 44us) client_negotiation.cc:240] Negotiation successful
0619 10:20:52.403098 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":332,"thread_start_us":61,"threads_started":1}
I20260619 10:20:52.403337 18765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:20:52.399192 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39478 (local address 127.0.206.66:36347)
0619 10:20:52.399357 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0619 10:20:52.399360 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 10:20:52.399466 (+ 106us) server_negotiation.cc:407] Connection header received
0619 10:20:52.399508 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:20:52.399510 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:20:52.399524 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:20:52.399610 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 10:20:52.399956 (+ 346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:20:52.401012 (+ 1056us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:20:52.402004 (+ 992us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:20:52.402717 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:20:52.402785 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:20:52.402871 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0619 10:20:52.403210 (+ 339us) server_negotiation.cc:299] Negotiation successful
0619 10:20:52.403253 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":127,"thread_start_us":95,"threads_started":1}
I20260619 10:20:52.403172 18762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:20:52.398943 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.66:36347 (local address 127.0.0.1:39478)
0619 10:20:52.399370 (+ 427us) negotiation.cc:107] Waiting for socket to connect
0619 10:20:52.399381 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 10:20:52.399441 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 10:20:52.399831 (+ 390us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 10:20:52.399832 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 10:20:52.399834 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 10:20:52.399869 (+ 35us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:20:52.399870 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:20:52.401073 (+ 1203us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:20:52.401074 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:20:52.401917 (+ 843us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:20:52.401919 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:20:52.402918 (+ 999us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:20:52.402919 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:20:52.402980 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:20:52.402984 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 10:20:52.403033 (+ 49us) client_negotiation.cc:240] Negotiation successful
0619 10:20:52.403078 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":410,"thread_start_us":111,"threads_started":1}
I20260619 10:20:52.403668 18766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:20:52.399205 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40166 (local address 127.0.206.65:42695)
0619 10:20:52.399452 (+ 247us) server_negotiation.cc:206] Beginning negotiation
0619 10:20:52.399455 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 10:20:52.399465 (+ 10us) server_negotiation.cc:407] Connection header received
0619 10:20:52.399507 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:20:52.399508 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:20:52.399524 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:20:52.399610 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 10:20:52.399818 (+ 208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:20:52.400926 (+ 1108us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:20:52.402101 (+ 1175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:20:52.402809 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:20:52.402864 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:20:52.402934 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0619 10:20:52.403511 (+ 577us) server_negotiation.cc:299] Negotiation successful
0619 10:20:52.403577 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":190,"thread_start_us":80,"threads_started":1}
I20260619 10:20:52.403807 18687 tablet_service.cc:1467] Processing CreateTablet for tablet 71e20148c8494a049b37272da6285b4f (DEFAULT_TABLE table=test-table [id=845bae5179294b03baf81a23d6a795d4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 10:20:52.404155 18687 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 71e20148c8494a049b37272da6285b4f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 10:20:52.404124 18556 tablet_service.cc:1467] Processing CreateTablet for tablet 71e20148c8494a049b37272da6285b4f (DEFAULT_TABLE table=test-table [id=845bae5179294b03baf81a23d6a795d4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 10:20:52.404428 18556 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 71e20148c8494a049b37272da6285b4f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 10:20:52.404387 18425 tablet_service.cc:1467] Processing CreateTablet for tablet 71e20148c8494a049b37272da6285b4f (DEFAULT_TABLE table=test-table [id=845bae5179294b03baf81a23d6a795d4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 10:20:52.404663 18425 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 71e20148c8494a049b37272da6285b4f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 10:20:52.406989 18767 tablet_bootstrap.cc:492] T 71e20148c8494a049b37272da6285b4f P 1b88d465d65c4954870bc4c8de4c74cb: Bootstrap starting.
I20260619 10:20:52.407728 18769 tablet_bootstrap.cc:492] T 71e20148c8494a049b37272da6285b4f P cb6d28bca45f4cb2b406e09691fa9f69: Bootstrap starting.
I20260619 10:20:52.407801 18767 tablet_bootstrap.cc:654] T 71e20148c8494a049b37272da6285b4f P 1b88d465d65c4954870bc4c8de4c74cb: Neither blocks nor log segments found. Creating new log.
I20260619 10:20:52.408077 18767 log.cc:824] T 71e20148c8494a049b37272da6285b4f P 1b88d465d65c4954870bc4c8de4c74cb: Log is configured to *not* fsync() on all Append() calls
I20260619 10:20:52.408344 18769 tablet_bootstrap.cc:654] T 71e20148c8494a049b37272da6285b4f P cb6d28bca45f4cb2b406e09691fa9f69: Neither blocks nor log segments found. Creating new log.
I20260619 10:20:52.408618 18769 log.cc:824] T 71e20148c8494a049b37272da6285b4f P cb6d28bca45f4cb2b406e09691fa9f69: Log is configured to *not* fsync() on all Append() calls
I20260619 10:20:52.408816 18767 tablet_bootstrap.cc:492] T 71e20148c8494a049b37272da6285b4f P 1b88d465d65c4954870bc4c8de4c74cb: No bootstrap required, opened a new log
I20260619 10:20:52.408885 18767 ts_tablet_manager.cc:1397] T 71e20148c8494a049b37272da6285b4f P 1b88d465d65c4954870bc4c8de4c74cb: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260619 10:20:52.408949 18768 tablet_bootstrap.cc:492] T 71e20148c8494a049b37272da6285b4f P c0f80e4da8354151a97f1bcfb156bfa8: Bootstrap starting.
I20260619 10:20:52.409511 18769 tablet_bootstrap.cc:492] T 71e20148c8494a049b37272da6285b4f P cb6d28bca45f4cb2b406e09691fa9f69: No bootstrap required, opened a new log
I20260619 10:20:52.409562 18769 ts_tablet_manager.cc:1397] T 71e20148c8494a049b37272da6285b4f P cb6d28bca45f4cb2b406e09691fa9f69: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260619 10:20:52.410969 18768 tablet_bootstrap.cc:654] T 71e20148c8494a049b37272da6285b4f P c0f80e4da8354151a97f1bcfb156bfa8: Neither blocks nor log segments found. Creating new log.
I20260619 10:20:52.410900 18767 raft_consensus.cc:348] T 71e20148c8494a049b37272da6285b4f P 1b88d465d65c4954870bc4c8de4c74cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0f80e4da8354151a97f1bcfb156bfa8" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 34987 } } peers { permanent_uuid: "cb6d28bca45f4cb2b406e09691fa9f69" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 36347 } } peers { permanent_uuid: "1b88d465d65c4954870bc4c8de4c74cb" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 42695 } }
I20260619 10:20:52.411036 18767 raft_consensus.cc:374] T 71e20148c8494a049b37272da6285b4f P 1b88d465d65c4954870bc4c8de4c74cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 10:20:52.411090 18767 raft_consensus.cc:729] T 71e20148c8494a049b37272da6285b4f P 1b88d465d65c4954870bc4c8de4c74cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b88d465d65c4954870bc4c8de4c74cb, State: Initialized, Role: FOLLOWER
I20260619 10:20:52.411209 18767 consensus_queue.cc:260] T 71e20148c8494a049b37272da6285b4f P 1b88d465d65c4954870bc4c8de4c74cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0f80e4da8354151a97f1bcfb156bfa8" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 34987 } } peers { permanent_uuid: "cb6d28bca45f4cb2b406e09691fa9f69" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 36347 } } peers { permanent_uuid: "1b88d465d65c4954870bc4c8de4c74cb" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 42695 } }
I20260619 10:20:52.411249 18768 log.cc:824] T 71e20148c8494a049b37272da6285b4f P c0f80e4da8354151a97f1bcfb156bfa8: Log is configured to *not* fsync() on all Append() calls
I20260619 10:20:52.411466 18767 ts_tablet_manager.cc:1428] T 71e20148c8494a049b37272da6285b4f P 1b88d465d65c4954870bc4c8de4c74cb: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260619 10:20:52.411485 18491 heartbeater.cc:503] Master 127.0.206.126:38687 was elected leader, sending a full tablet report...
I20260619 10:20:52.411674 18769 raft_consensus.cc:348] T 71e20148c8494a049b37272da6285b4f P cb6d28bca45f4cb2b406e09691fa9f69 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0f80e4da8354151a97f1bcfb156bfa8" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 34987 } } peers { permanent_uuid: "cb6d28bca45f4cb2b406e09691fa9f69" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 36347 } } peers { permanent_uuid: "1b88d465d65c4954870bc4c8de4c74cb" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 42695 } }
I20260619 10:20:52.411818 18769 raft_consensus.cc:374] T 71e20148c8494a049b37272da6285b4f P cb6d28bca45f4cb2b406e09691fa9f69 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 10:20:52.411850 18769 raft_consensus.cc:729] T 71e20148c8494a049b37272da6285b4f P cb6d28bca45f4cb2b406e09691fa9f69 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cb6d28bca45f4cb2b406e09691fa9f69, State: Initialized, Role: FOLLOWER
I20260619 10:20:52.411980 18769 consensus_queue.cc:260] T 71e20148c8494a049b37272da6285b4f P cb6d28bca45f4cb2b406e09691fa9f69 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0f80e4da8354151a97f1bcfb156bfa8" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 34987 } } peers { permanent_uuid: "cb6d28bca45f4cb2b406e09691fa9f69" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 36347 } } peers { permanent_uuid: "1b88d465d65c4954870bc4c8de4c74cb" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 42695 } }
I20260619 10:20:52.412225 18768 tablet_bootstrap.cc:492] T 71e20148c8494a049b37272da6285b4f P c0f80e4da8354151a97f1bcfb156bfa8: No bootstrap required, opened a new log
I20260619 10:20:52.412233 18769 ts_tablet_manager.cc:1428] T 71e20148c8494a049b37272da6285b4f P cb6d28bca45f4cb2b406e09691fa9f69: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260619 10:20:52.412333 18622 heartbeater.cc:503] Master 127.0.206.126:38687 was elected leader, sending a full tablet report...
I20260619 10:20:52.412345 18768 ts_tablet_manager.cc:1397] T 71e20148c8494a049b37272da6285b4f P c0f80e4da8354151a97f1bcfb156bfa8: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260619 10:20:52.414369 18768 raft_consensus.cc:348] T 71e20148c8494a049b37272da6285b4f P c0f80e4da8354151a97f1bcfb156bfa8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0f80e4da8354151a97f1bcfb156bfa8" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 34987 } } peers { permanent_uuid: "cb6d28bca45f4cb2b406e09691fa9f69" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 36347 } } peers { permanent_uuid: "1b88d465d65c4954870bc4c8de4c74cb" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 42695 } }
I20260619 10:20:52.414503 18768 raft_consensus.cc:374] T 71e20148c8494a049b37272da6285b4f P c0f80e4da8354151a97f1bcfb156bfa8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 10:20:52.414547 18768 raft_consensus.cc:729] T 71e20148c8494a049b37272da6285b4f P c0f80e4da8354151a97f1bcfb156bfa8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c0f80e4da8354151a97f1bcfb156bfa8, State: Initialized, Role: FOLLOWER
I20260619 10:20:52.414667 18768 consensus_queue.cc:260] T 71e20148c8494a049b37272da6285b4f P c0f80e4da8354151a97f1bcfb156bfa8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0f80e4da8354151a97f1bcfb156bfa8" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 34987 } } peers { permanent_uuid: "cb6d28bca45f4cb2b406e09691fa9f69" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 36347 } } peers { permanent_uuid: "1b88d465d65c4954870bc4c8de4c74cb" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 42695 } }
I20260619 10:20:52.414906 18768 ts_tablet_manager.cc:1428] T 71e20148c8494a049b37272da6285b4f P c0f80e4da8354151a97f1bcfb156bfa8: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260619 10:20:52.414986 18753 heartbeater.cc:503] Master 127.0.206.126:38687 was elected leader, sending a full tablet report...
I20260619 10:20:52.452275 18772 raft_consensus.cc:482] T 71e20148c8494a049b37272da6285b4f P 1b88d465d65c4954870bc4c8de4c74cb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 10:20:52.452369 18772 raft_consensus.cc:504] T 71e20148c8494a049b37272da6285b4f P 1b88d465d65c4954870bc4c8de4c74cb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0f80e4da8354151a97f1bcfb156bfa8" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 34987 } } peers { permanent_uuid: "cb6d28bca45f4cb2b406e09691fa9f69" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 36347 } } peers { permanent_uuid: "1b88d465d65c4954870bc4c8de4c74cb" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 42695 } }
I20260619 10:20:52.452698 18772 leader_election.cc:288] T 71e20148c8494a049b37272da6285b4f P 1b88d465d65c4954870bc4c8de4c74cb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c0f80e4da8354151a97f1bcfb156bfa8 (127.0.206.67:34987), cb6d28bca45f4cb2b406e09691fa9f69 (127.0.206.66:36347)
I20260619 10:20:52.456414 18764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:20:52.452939 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.65:42945 (local address 127.0.206.67:34987)
0619 10:20:52.452983 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0619 10:20:52.452985 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 10:20:52.452995 (+ 10us) server_negotiation.cc:407] Connection header received
0619 10:20:52.453048 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:20:52.453049 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:20:52.453059 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:20:52.453134 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 10:20:52.453309 (+ 175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:20:52.454071 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:20:52.455561 (+ 1490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:20:52.456068 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:20:52.456131 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:20:52.456138 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0619 10:20:52.456304 (+ 166us) server_negotiation.cc:299] Negotiation successful
0619 10:20:52.456337 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260619 10:20:52.456420 18765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:20:52.452993 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.65:60223 (local address 127.0.206.66:36347)
0619 10:20:52.453045 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0619 10:20:52.453047 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 10:20:52.453181 (+ 134us) server_negotiation.cc:407] Connection header received
0619 10:20:52.453247 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:20:52.453248 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:20:52.453255 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:20:52.453321 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 10:20:52.453451 (+ 130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:20:52.454117 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:20:52.455559 (+ 1442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:20:52.456058 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:20:52.456131 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:20:52.456138 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0619 10:20:52.456304 (+ 166us) server_negotiation.cc:299] Negotiation successful
0619 10:20:52.456352 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260619 10:20:52.456444 18493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:20:52.452857 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.67:34987 (local address 127.0.206.65:42945)
0619 10:20:52.452924 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0619 10:20:52.452936 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 10:20:52.452982 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 10:20:52.453158 (+ 176us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 10:20:52.453159 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 10:20:52.453160 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 10:20:52.453237 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:20:52.453239 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:20:52.454165 (+ 926us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:20:52.454166 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:20:52.455474 (+ 1308us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:20:52.455477 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:20:52.456143 (+ 666us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:20:52.456144 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:20:52.456229 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:20:52.456235 (+ 6us) client_negotiation.cc:769] Sending connection context
0619 10:20:52.456287 (+ 52us) client_negotiation.cc:240] Negotiation successful
0619 10:20:52.456324 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":37,"mutex_wait_us":46}
I20260619 10:20:52.456449 18776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:20:52.452951 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.66:36347 (local address 127.0.206.65:60223)
0619 10:20:52.453129 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0619 10:20:52.453138 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0619 10:20:52.453188 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 10:20:52.453335 (+ 147us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 10:20:52.453336 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 10:20:52.453337 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 10:20:52.453392 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:20:52.453393 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:20:52.454182 (+ 789us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:20:52.454182 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:20:52.455474 (+ 1292us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:20:52.455477 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:20:52.456145 (+ 668us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:20:52.456145 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:20:52.456229 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:20:52.456235 (+ 6us) client_negotiation.cc:769] Sending connection context
0619 10:20:52.456287 (+ 52us) client_negotiation.cc:240] Negotiation successful
0619 10:20:52.456324 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":152,"mutex_wait_us":47,"thread_start_us":104,"threads_started":1}
I20260619 10:20:52.456761 18707 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "71e20148c8494a049b37272da6285b4f" candidate_uuid: "1b88d465d65c4954870bc4c8de4c74cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c0f80e4da8354151a97f1bcfb156bfa8" is_pre_election: true
I20260619 10:20:52.456845 18576 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "71e20148c8494a049b37272da6285b4f" candidate_uuid: "1b88d465d65c4954870bc4c8de4c74cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb6d28bca45f4cb2b406e09691fa9f69" is_pre_election: true
I20260619 10:20:52.456912 18707 raft_consensus.cc:2454] T 71e20148c8494a049b37272da6285b4f P c0f80e4da8354151a97f1bcfb156bfa8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1b88d465d65c4954870bc4c8de4c74cb in term 0.
I20260619 10:20:52.456988 18576 raft_consensus.cc:2454] T 71e20148c8494a049b37272da6285b4f P cb6d28bca45f4cb2b406e09691fa9f69 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1b88d465d65c4954870bc4c8de4c74cb in term 0.
I20260619 10:20:52.457160 18381 leader_election.cc:302] T 71e20148c8494a049b37272da6285b4f P 1b88d465d65c4954870bc4c8de4c74cb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1b88d465d65c4954870bc4c8de4c74cb, c0f80e4da8354151a97f1bcfb156bfa8; no voters:
I20260619 10:20:52.457381 18772 raft_consensus.cc:2781] T 71e20148c8494a049b37272da6285b4f P 1b88d465d65c4954870bc4c8de4c74cb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 10:20:52.457420 18772 raft_consensus.cc:482] T 71e20148c8494a049b37272da6285b4f P 1b88d465d65c4954870bc4c8de4c74cb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 10:20:52.457435 18772 raft_consensus.cc:3037] T 71e20148c8494a049b37272da6285b4f P 1b88d465d65c4954870bc4c8de4c74cb [term 0 FOLLOWER]: Advancing to term 1
I20260619 10:20:52.458060 18772 raft_consensus.cc:504] T 71e20148c8494a049b37272da6285b4f P 1b88d465d65c4954870bc4c8de4c74cb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0f80e4da8354151a97f1bcfb156bfa8" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 34987 } } peers { permanent_uuid: "cb6d28bca45f4cb2b406e09691fa9f69" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 36347 } } peers { permanent_uuid: "1b88d465d65c4954870bc4c8de4c74cb" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 42695 } }
I20260619 10:20:52.458191 18772 leader_election.cc:288] T 71e20148c8494a049b37272da6285b4f P 1b88d465d65c4954870bc4c8de4c74cb [CANDIDATE]: Term 1 election: Requested vote from peers c0f80e4da8354151a97f1bcfb156bfa8 (127.0.206.67:34987), cb6d28bca45f4cb2b406e09691fa9f69 (127.0.206.66:36347)
I20260619 10:20:52.458324 18707 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "71e20148c8494a049b37272da6285b4f" candidate_uuid: "1b88d465d65c4954870bc4c8de4c74cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c0f80e4da8354151a97f1bcfb156bfa8"
I20260619 10:20:52.458341 18576 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "71e20148c8494a049b37272da6285b4f" candidate_uuid: "1b88d465d65c4954870bc4c8de4c74cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb6d28bca45f4cb2b406e09691fa9f69"
I20260619 10:20:52.458403 18707 raft_consensus.cc:3037] T 71e20148c8494a049b37272da6285b4f P c0f80e4da8354151a97f1bcfb156bfa8 [term 0 FOLLOWER]: Advancing to term 1
I20260619 10:20:52.458415 18576 raft_consensus.cc:3037] T 71e20148c8494a049b37272da6285b4f P cb6d28bca45f4cb2b406e09691fa9f69 [term 0 FOLLOWER]: Advancing to term 1
I20260619 10:20:52.459297 18576 raft_consensus.cc:2454] T 71e20148c8494a049b37272da6285b4f P cb6d28bca45f4cb2b406e09691fa9f69 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1b88d465d65c4954870bc4c8de4c74cb in term 1.
I20260619 10:20:52.459297 18707 raft_consensus.cc:2454] T 71e20148c8494a049b37272da6285b4f P c0f80e4da8354151a97f1bcfb156bfa8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1b88d465d65c4954870bc4c8de4c74cb in term 1.
I20260619 10:20:52.459462 18378 leader_election.cc:302] T 71e20148c8494a049b37272da6285b4f P 1b88d465d65c4954870bc4c8de4c74cb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1b88d465d65c4954870bc4c8de4c74cb, cb6d28bca45f4cb2b406e09691fa9f69; no voters:
I20260619 10:20:52.459568 18772 raft_consensus.cc:2781] T 71e20148c8494a049b37272da6285b4f P 1b88d465d65c4954870bc4c8de4c74cb [term 1 FOLLOWER]: Leader election won for term 1
I20260619 10:20:52.459702 18772 raft_consensus.cc:686] T 71e20148c8494a049b37272da6285b4f P 1b88d465d65c4954870bc4c8de4c74cb [term 1 LEADER]: Becoming Leader. State: Replica: 1b88d465d65c4954870bc4c8de4c74cb, State: Running, Role: LEADER
I20260619 10:20:52.459789 18772 consensus_queue.cc:237] T 71e20148c8494a049b37272da6285b4f P 1b88d465d65c4954870bc4c8de4c74cb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0f80e4da8354151a97f1bcfb156bfa8" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 34987 } } peers { permanent_uuid: "cb6d28bca45f4cb2b406e09691fa9f69" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 36347 } } peers { permanent_uuid: "1b88d465d65c4954870bc4c8de4c74cb" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 42695 } }
I20260619 10:20:52.460562 18302 catalog_manager.cc:5495] T 71e20148c8494a049b37272da6285b4f P 1b88d465d65c4954870bc4c8de4c74cb reported cstate change: term changed from 0 to 1, leader changed from <none> to 1b88d465d65c4954870bc4c8de4c74cb (127.0.206.65). New cstate: current_term: 1 leader_uuid: "1b88d465d65c4954870bc4c8de4c74cb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0f80e4da8354151a97f1bcfb156bfa8" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 34987 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb6d28bca45f4cb2b406e09691fa9f69" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 36347 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1b88d465d65c4954870bc4c8de4c74cb" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 42695 } health_report { overall_health: HEALTHY } } }
I20260619 10:20:52.477506 18766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:20:52.475546 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40180 (local address 127.0.206.65:42695)
0619 10:20:52.475600 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0619 10:20:52.475601 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 10:20:52.475608 (+ 7us) server_negotiation.cc:407] Connection header received
0619 10:20:52.475634 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:20:52.475635 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:20:52.475649 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:20:52.475703 (+ 54us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 10:20:52.475807 (+ 104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:20:52.476215 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:20:52.476830 (+ 615us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:20:52.477005 (+ 175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:20:52.477050 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:20:52.477165 (+ 115us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 10:20:52.477246 (+ 81us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 10:20:52.477331 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0619 10:20:52.477372 (+ 41us) server_negotiation.cc:299] Negotiation successful
0619 10:20:52.477386 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260619 10:20:52.479143 18707 raft_consensus.cc:1261] T 71e20148c8494a049b37272da6285b4f P c0f80e4da8354151a97f1bcfb156bfa8 [term 1 FOLLOWER]: Refusing update from remote peer 1b88d465d65c4954870bc4c8de4c74cb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 10:20:52.479148 18576 raft_consensus.cc:1261] T 71e20148c8494a049b37272da6285b4f P cb6d28bca45f4cb2b406e09691fa9f69 [term 1 FOLLOWER]: Refusing update from remote peer 1b88d465d65c4954870bc4c8de4c74cb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 10:20:52.479478 18772 consensus_queue.cc:1035] T 71e20148c8494a049b37272da6285b4f P 1b88d465d65c4954870bc4c8de4c74cb [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb6d28bca45f4cb2b406e09691fa9f69" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 36347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 10:20:52.479625 18777 consensus_queue.cc:1035] T 71e20148c8494a049b37272da6285b4f P 1b88d465d65c4954870bc4c8de4c74cb [LEADER]: Connected to new peer: Peer: permanent_uuid: "c0f80e4da8354151a97f1bcfb156bfa8" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 34987 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 10:20:52.481832 18782 mvcc.cc:205] Tried to move back new op lower bound from 7298516797351911424 to 7298516797275774976. Current Snapshot: MvccSnapshot[applied={T|T < 7298516797351911424}]
I20260619 10:20:52.481842 18781 mvcc.cc:205] Tried to move back new op lower bound from 7298516797351911424 to 7298516797275774976. Current Snapshot: MvccSnapshot[applied={T|T < 7298516797351911424}]
I20260619 10:20:52.484076 18303 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:37254:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 10:20:52.484157 18303 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:37254:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 10:20:52.484849 18303 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 93286777d79d4689858b23f69f34c428: Sending DeleteTablet for 3 replicas of tablet 71e20148c8494a049b37272da6285b4f
I20260619 10:20:52.485164 18556 tablet_service.cc:1514] Processing DeleteTablet for tablet 71e20148c8494a049b37272da6285b4f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 10:20:52 UTC) from {username='slave'} at 127.0.0.1:39478
I20260619 10:20:52.485208 18687 tablet_service.cc:1514] Processing DeleteTablet for tablet 71e20148c8494a049b37272da6285b4f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 10:20:52 UTC) from {username='slave'} at 127.0.0.1:42830
I20260619 10:20:52.485360 18425 tablet_service.cc:1514] Processing DeleteTablet for tablet 71e20148c8494a049b37272da6285b4f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 10:20:52 UTC) from {username='slave'} at 127.0.0.1:40166
I20260619 10:20:52.485531 825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTU9OP6/build/release/bin/kudu with pid 18363
I20260619 10:20:52.485610 18791 tablet_replica.cc:330] T 71e20148c8494a049b37272da6285b4f P c0f80e4da8354151a97f1bcfb156bfa8: stopping tablet replica
I20260619 10:20:52.486248 18791 raft_consensus.cc:2229] T 71e20148c8494a049b37272da6285b4f P c0f80e4da8354151a97f1bcfb156bfa8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 10:20:52.486335 18791 raft_consensus.cc:2258] T 71e20148c8494a049b37272da6285b4f P c0f80e4da8354151a97f1bcfb156bfa8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 10:20:52.486501 18790 tablet_replica.cc:330] T 71e20148c8494a049b37272da6285b4f P cb6d28bca45f4cb2b406e09691fa9f69: stopping tablet replica
I20260619 10:20:52.486647 18790 raft_consensus.cc:2229] T 71e20148c8494a049b37272da6285b4f P cb6d28bca45f4cb2b406e09691fa9f69 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 10:20:52.486785 18790 raft_consensus.cc:2258] T 71e20148c8494a049b37272da6285b4f P cb6d28bca45f4cb2b406e09691fa9f69 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 10:20:52.486786 18791 ts_tablet_manager.cc:1905] T 71e20148c8494a049b37272da6285b4f P c0f80e4da8354151a97f1bcfb156bfa8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 10:20:52.487231 18790 ts_tablet_manager.cc:1905] T 71e20148c8494a049b37272da6285b4f P cb6d28bca45f4cb2b406e09691fa9f69: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 10:20:52.488456 18791 ts_tablet_manager.cc:1918] T 71e20148c8494a049b37272da6285b4f P c0f80e4da8354151a97f1bcfb156bfa8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 10:20:52.488523 18791 log.cc:1196] T 71e20148c8494a049b37272da6285b4f P c0f80e4da8354151a97f1bcfb156bfa8: Deleting WAL directory at /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781864224577058-825-0/minicluster-data/ts-2/wal/wals/71e20148c8494a049b37272da6285b4f
I20260619 10:20:52.488776 18791 ts_tablet_manager.cc:1939] T 71e20148c8494a049b37272da6285b4f P c0f80e4da8354151a97f1bcfb156bfa8: Deleting consensus metadata
I20260619 10:20:52.488780 18790 ts_tablet_manager.cc:1918] T 71e20148c8494a049b37272da6285b4f P cb6d28bca45f4cb2b406e09691fa9f69: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 10:20:52.488834 18790 log.cc:1196] T 71e20148c8494a049b37272da6285b4f P cb6d28bca45f4cb2b406e09691fa9f69: Deleting WAL directory at /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781864224577058-825-0/minicluster-data/ts-1/wal/wals/71e20148c8494a049b37272da6285b4f
I20260619 10:20:52.489153 18790 ts_tablet_manager.cc:1939] T 71e20148c8494a049b37272da6285b4f P cb6d28bca45f4cb2b406e09691fa9f69: Deleting consensus metadata
I20260619 10:20:52.489223 18290 catalog_manager.cc:4841] TS c0f80e4da8354151a97f1bcfb156bfa8 (127.0.206.67:34987): tablet 71e20148c8494a049b37272da6285b4f (table test-table [id=845bae5179294b03baf81a23d6a795d4]) successfully deleted
I20260619 10:20:52.489526 18287 catalog_manager.cc:4841] TS cb6d28bca45f4cb2b406e09691fa9f69 (127.0.206.66:36347): tablet 71e20148c8494a049b37272da6285b4f (table test-table [id=845bae5179294b03baf81a23d6a795d4]) successfully deleted
W20260619 10:20:52.490626 18288 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.0.206.65:42695 (error 108)
I20260619 10:20:52.491015 18762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:20:52.490910 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.65:42695 (local address 127.0.0.1:40190)
0619 10:20:52.490960 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0619 10:20:52.490993 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.65:42695: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260619 10:20:52.491053 825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTU9OP6/build/release/bin/kudu with pid 18494
W20260619 10:20:52.491143 18288 catalog_manager.cc:4584] TS 1b88d465d65c4954870bc4c8de4c74cb (127.0.206.65:42695): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 71e20148c8494a049b37272da6285b4f: Network error: Client connection negotiation failed: client connection to 127.0.206.65:42695: connect: Connection refused (error 111)
I20260619 10:20:52.495432 825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTU9OP6/build/release/bin/kudu with pid 18625
I20260619 10:20:52.499559 825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTU9OP6/build/release/bin/kudu with pid 18272
2026-06-19T10:20:52Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (630 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781864224577058-825-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 10:20:52 dist-test-slave-6pt2 krb5kdc[18796](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 10:20:52 dist-test-slave-6pt2 krb5kdc[18796](info): set up 2 sockets
Jun 19 10:20:52 dist-test-slave-6pt2 krb5kdc[18796](info): commencing operation
krb5kdc: starting...
W20260619 10:20:53.510263 825 mini_kdc.cc:121] Time spent starting KDC: real 0.992s user 0.005s sys 0.000s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781864224577058-825-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781864224577058-825-0/krb5kdc/test-user.keytab.
Jun 19 10:20:53 dist-test-slave-6pt2 krb5kdc[18796](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864453, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T10:20:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T10:20:53Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.206.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.206.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.206.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781864224577058-825-0/krb5kdc/kudu_127.0.206.126.keytab.
Entry for principal kudu/127.0.206.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781864224577058-825-0/krb5kdc/kudu_127.0.206.126.keytab.
I20260619 10:20:53.616627 825 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTU9OP6/build/release/bin/kudu
/tmp/dist-test-taskTU9OP6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781864224577058-825-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781864224577058-825-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781864224577058-825-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781864224577058-825-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.206.126:39125
--webserver_interface=127.0.206.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781864224577058-825-0/krb5kdc/kudu_127.0.206.126.keytab
--principal=kudu/127.0.206.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.206.84:44173
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.206.126:39125
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781864224577058-825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781864224577058-825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781864224577058-825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 10:20:53.693003 18809 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 10:20:53.693245 18809 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 10:20:53.693282 18809 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 10:20:53.694537 18809 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 10:20:53.694608 18809 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 10:20:53.694672 18809 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 10:20:53.694701 18809 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 10:20:53.694729 18809 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 10:20:53.696158 18809 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.206.84:44173
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781864224577058-825-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781864224577058-825-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.206.126:39125
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781864224577058-825-0/krb5kdc/kudu_127.0.206.126.keytab
--principal=kudu/127.0.206.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.206.126:39125
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781864224577058-825-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.206.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18809
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781864224577058-825-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 09:32:44 UTC on ccycloud.quasar-gbtldp.root.comops.site
build id 4455440
I20260619 10:20:53.696424 18809 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 10:20:53.696645 18809 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 10:20:53.701337 18814 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 10:20:53.701339 18817 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 10:20:53.701365 18809 server_base.cc:1034] running on GCE node
W20260619 10:20:53.701354 18815 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 10:20:53.701834 18809 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 10:20:53.702081 18809 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 10:20:53.703225 18809 hybrid_clock.cc:648] HybridClock initialized: now 1781864453703216 us; error 30 us; skew 500 ppm
Jun 19 10:20:53 dist-test-slave-6pt2 krb5kdc[18796](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864453, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 10:20:53.710021 18809 init.cc:377] Logged in from keytab as kudu/127.0.206.126@KRBTEST.COM (short username kudu)
I20260619 10:20:53.710456 18809 webserver.cc:460] Webserver started at http://127.0.206.126:38245/ using document root <none> and password file <none>
I20260619 10:20:53.710749 18809 fs_manager.cc:359] Metadata directory not provided
I20260619 10:20:53.710795 18809 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 10:20:53.710937 18809 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 10:20:53.711946 18809 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781864224577058-825-0/minicluster-data/master-0/data/instance:
uuid: "52c1c2a8b1ea4577b1b7fb68234b509e"
format_stamp: "Formatted at 2026-06-19 10:20:53 on dist-test-slave-6pt2"
server_key: "c530e9e79b398c1b104eea90febd4f75"
server_key_iv: "3c5ecedbb3e22433432af26aa5bb02e3"
server_key_version: "encryptionkey@0"
I20260619 10:20:53.712291 18809 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781864224577058-825-0/minicluster-data/master-0/wal/instance:
uuid: "52c1c2a8b1ea4577b1b7fb68234b509e"
format_stamp: "Formatted at 2026-06-19 10:20:53 on dist-test-slave-6pt2"
server_key: "c530e9e79b398c1b104eea90febd4f75"
server_key_iv: "3c5ecedbb3e22433432af26aa5bb02e3"
server_key_version: "encryptionkey@0"
I20260619 10:20:53.713567 18809 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 10:20:53.714257 18824 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 10:20:53.714433 18809 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 10:20:53.714530 18809 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781864224577058-825-0/minicluster-data/master-0/data,/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781864224577058-825-0/minicluster-data/master-0/wal
uuid: "52c1c2a8b1ea4577b1b7fb68234b509e"
format_stamp: "Formatted at 2026-06-19 10:20:53 on dist-test-slave-6pt2"
server_key: "c530e9e79b398c1b104eea90febd4f75"
server_key_iv: "3c5ecedbb3e22433432af26aa5bb02e3"
server_key_version: "encryptionkey@0"
I20260619 10:20:53.714635 18809 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781864224577058-825-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781864224577058-825-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781864224577058-825-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 10:20:53.734489 18809 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 10:20:53.736755 18809 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 10:20:53.736914 18809 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 10:20:53.740617 18809 rpc_server.cc:307] RPC server started. Bound to: 127.0.206.126:39125
I20260619 10:20:53.740662 18876 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.206.126:39125 every 8 connection(s)
I20260619 10:20:53.740988 18809 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781864224577058-825-0/minicluster-data/master-0/data/info.pb
I20260619 10:20:53.741767 18877 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 10:20:53.743959 18877 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 52c1c2a8b1ea4577b1b7fb68234b509e: Bootstrap starting.
I20260619 10:20:53.744515 18877 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 52c1c2a8b1ea4577b1b7fb68234b509e: Neither blocks nor log segments found. Creating new log.
I20260619 10:20:53.744784 18877 log.cc:824] T 00000000000000000000000000000000 P 52c1c2a8b1ea4577b1b7fb68234b509e: Log is configured to *not* fsync() on all Append() calls
I20260619 10:20:53.745455 18877 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 52c1c2a8b1ea4577b1b7fb68234b509e: No bootstrap required, opened a new log
I20260619 10:20:53.746991 18877 raft_consensus.cc:348] T 00000000000000000000000000000000 P 52c1c2a8b1ea4577b1b7fb68234b509e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "52c1c2a8b1ea4577b1b7fb68234b509e" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 39125 } }
I20260619 10:20:53.747139 18877 raft_consensus.cc:374] T 00000000000000000000000000000000 P 52c1c2a8b1ea4577b1b7fb68234b509e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 10:20:53.747187 18877 raft_consensus.cc:729] T 00000000000000000000000000000000 P 52c1c2a8b1ea4577b1b7fb68234b509e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 52c1c2a8b1ea4577b1b7fb68234b509e, State: Initialized, Role: FOLLOWER
I20260619 10:20:53.747344 18877 consensus_queue.cc:260] T 00000000000000000000000000000000 P 52c1c2a8b1ea4577b1b7fb68234b509e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "52c1c2a8b1ea4577b1b7fb68234b509e" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 39125 } }
I20260619 10:20:53.747431 18877 raft_consensus.cc:388] T 00000000000000000000000000000000 P 52c1c2a8b1ea4577b1b7fb68234b509e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 10:20:53.747471 18877 raft_consensus.cc:482] T 00000000000000000000000000000000 P 52c1c2a8b1ea4577b1b7fb68234b509e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 10:20:53.747520 18877 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 52c1c2a8b1ea4577b1b7fb68234b509e [term 0 FOLLOWER]: Advancing to term 1
I20260619 10:20:53.748015 18877 raft_consensus.cc:504] T 00000000000000000000000000000000 P 52c1c2a8b1ea4577b1b7fb68234b509e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "52c1c2a8b1ea4577b1b7fb68234b509e" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 39125 } }
I20260619 10:20:53.748126 18877 leader_election.cc:302] T 00000000000000000000000000000000 P 52c1c2a8b1ea4577b1b7fb68234b509e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 52c1c2a8b1ea4577b1b7fb68234b509e; no voters:
I20260619 10:20:53.748299 18877 leader_election.cc:288] T 00000000000000000000000000000000 P 52c1c2a8b1ea4577b1b7fb68234b509e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 10:20:53.748334 18880 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 52c1c2a8b1ea4577b1b7fb68234b509e [term 1 FOLLOWER]: Leader election won for term 1
I20260619 10:20:53.748490 18880 raft_consensus.cc:686] T 00000000000000000000000000000000 P 52c1c2a8b1ea4577b1b7fb68234b509e [term 1 LEADER]: Becoming Leader. State: Replica: 52c1c2a8b1ea4577b1b7fb68234b509e, State: Running, Role: LEADER
I20260619 10:20:53.748558 18877 sys_catalog.cc:564] T 00000000000000000000000000000000 P 52c1c2a8b1ea4577b1b7fb68234b509e [sys.catalog]: configured and running, proceeding with master startup.
I20260619 10:20:53.748629 18880 consensus_queue.cc:237] T 00000000000000000000000000000000 P 52c1c2a8b1ea4577b1b7fb68234b509e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "52c1c2a8b1ea4577b1b7fb68234b509e" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 39125 } }
I20260619 10:20:53.748989 18882 sys_catalog.cc:455] T 00000000000000000000000000000000 P 52c1c2a8b1ea4577b1b7fb68234b509e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 52c1c2a8b1ea4577b1b7fb68234b509e. Latest consensus state: current_term: 1 leader_uuid: "52c1c2a8b1ea4577b1b7fb68234b509e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "52c1c2a8b1ea4577b1b7fb68234b509e" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 39125 } } }
I20260619 10:20:53.749070 18882 sys_catalog.cc:458] T 00000000000000000000000000000000 P 52c1c2a8b1ea4577b1b7fb68234b509e [sys.catalog]: This master's current role is: LEADER
I20260619 10:20:53.749106 18881 sys_catalog.cc:455] T 00000000000000000000000000000000 P 52c1c2a8b1ea4577b1b7fb68234b509e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "52c1c2a8b1ea4577b1b7fb68234b509e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "52c1c2a8b1ea4577b1b7fb68234b509e" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 39125 } } }
I20260619 10:20:53.749161 18881 sys_catalog.cc:458] T 00000000000000000000000000000000 P 52c1c2a8b1ea4577b1b7fb68234b509e [sys.catalog]: This master's current role is: LEADER
I20260619 10:20:53.749282 18888 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 10:20:53.749830 18888 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 10:20:53.751009 825 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTU9OP6/build/release/bin/kudu as pid 18809
I20260619 10:20:53.751119 825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781864224577058-825-0/minicluster-data/master-0/wal/instance
I20260619 10:20:53.751271 825 external_mini_cluster.cc:1428] Setting key ef1ac3cdb113a6313a64c0bad497655f
I20260619 10:20:53.751407 18888 catalog_manager.cc:1347] Generated new cluster ID: 80e59d63dec540fdb1077796804f08c3
I20260619 10:20:53.751451 18888 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 10:20:53.762493 18888 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 10:20:53.763234 18888 catalog_manager.cc:1504] Loading token signing keys...
Jun 19 10:20:53 dist-test-slave-6pt2 krb5kdc[18796](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864453, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.206.126@KRBTEST.COM
I20260619 10:20:53.767727 18900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:20:53.751696 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58442 (local address 127.0.206.126:39125)
0619 10:20:53.759000 (+ 7304us) server_negotiation.cc:206] Beginning negotiation
0619 10:20:53.759008 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 10:20:53.759031 (+ 23us) server_negotiation.cc:407] Connection header received
0619 10:20:53.759120 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:20:53.759124 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:20:53.759171 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:20:53.759303 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:20:53.759933 (+ 630us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:20:53.760845 (+ 912us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:20:53.761584 (+ 739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:20:53.762074 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:20:53.762208 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:20:53.764594 (+ 2386us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 10:20:53.764599 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 10:20:53.764600 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 10:20:53.764615 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0619 10:20:53.766200 (+ 1585us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:20:53.766619 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:20:53.766623 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:20:53.766623 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:20:53.766714 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:20:53.766931 (+ 217us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:20:53.766932 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:20:53.766933 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:20:53.767136 (+ 203us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 10:20:53.767284 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0619 10:20:53.767398 (+ 114us) server_negotiation.cc:299] Negotiation successful
0619 10:20:53.767553 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7267,"thread_start_us":63,"threads_started":1}
I20260619 10:20:53.778067 18888 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 52c1c2a8b1ea4577b1b7fb68234b509e: Generated new TSK 0
I20260619 10:20:53.778246 18888 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.0.206.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.206.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.206.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781864224577058-825-0/krb5kdc/kudu_127.0.206.65.keytab.
Entry for principal kudu/127.0.206.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781864224577058-825-0/krb5kdc/kudu_127.0.206.65.keytab.
I20260619 10:20:53.807330 825 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTU9OP6/build/release/bin/kudu
/tmp/dist-test-taskTU9OP6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781864224577058-825-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781864224577058-825-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781864224577058-825-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781864224577058-825-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.206.65:0
--local_ip_for_outbound_sockets=127.0.206.65
--webserver_interface=127.0.206.65
--webserver_port=0
--tserver_master_addrs=127.0.206.126:39125
--keytab_file=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781864224577058-825-0/krb5kdc/kudu_127.0.206.65.keytab
--principal=kudu/127.0.206.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.206.84:44173
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781864224577058-825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781864224577058-825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781864224577058-825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 10:20:53.907599 18903 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 10:20:53.907843 18903 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 10:20:53.907883 18903 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 10:20:53.909051 18903 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 10:20:53.909127 18903 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 10:20:53.909173 18903 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.206.65
I20260619 10:20:53.910719 18903 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.206.84:44173
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781864224577058-825-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781864224577058-825-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781864224577058-825-0/krb5kdc/kudu_127.0.206.65.keytab
--principal=kudu/127.0.206.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.206.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781864224577058-825-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.206.65
--webserver_port=0
--tserver_master_addrs=127.0.206.126:39125
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18903
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.206.65
--log_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781864224577058-825-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 09:32:44 UTC on ccycloud.quasar-gbtldp.root.comops.site
build id 4455440
I20260619 10:20:53.910967 18903 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 10:20:53.911185 18903 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 10:20:53.916155 18908 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 10:20:53.916143 18909 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 10:20:53.916143 18911 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 10:20:53.916172 18903 server_base.cc:1034] running on GCE node
I20260619 10:20:53.916633 18903 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 10:20:53.916874 18903 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 10:20:53.918057 18903 hybrid_clock.cc:648] HybridClock initialized: now 1781864453918040 us; error 33 us; skew 500 ppm
Jun 19 10:20:53 dist-test-slave-6pt2 krb5kdc[18796](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864453, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 10:20:53.920591 18903 init.cc:377] Logged in from keytab as kudu/127.0.206.65@KRBTEST.COM (short username kudu)
I20260619 10:20:53.921005 18903 webserver.cc:460] Webserver started at http://127.0.206.65:36267/ using document root <none> and password file <none>
I20260619 10:20:53.921219 18903 fs_manager.cc:359] Metadata directory not provided
I20260619 10:20:53.921285 18903 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 10:20:53.921434 18903 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 10:20:53.922428 18903 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781864224577058-825-0/minicluster-data/ts-0/data/instance:
uuid: "e3844982d2d44e3285b8408e1448d94e"
format_stamp: "Formatted at 2026-06-19 10:20:53 on dist-test-slave-6pt2"
server_key: "4ec0b34e3aaf5db811e35c0316eb2294"
server_key_iv: "c528b6932f62714a6a7297ccad8219e5"
server_key_version: "encryptionkey@0"
I20260619 10:20:53.922794 18903 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781864224577058-825-0/minicluster-data/ts-0/wal/instance:
uuid: "e3844982d2d44e3285b8408e1448d94e"
format_stamp: "Formatted at 2026-06-19 10:20:53 on dist-test-slave-6pt2"
server_key: "4ec0b34e3aaf5db811e35c0316eb2294"
server_key_iv: "c528b6932f62714a6a7297ccad8219e5"
server_key_version: "encryptionkey@0"
I20260619 10:20:53.924029 18903 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260619 10:20:53.924681 18918 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 10:20:53.924855 18903 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260619 10:20:53.924949 18903 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781864224577058-825-0/minicluster-data/ts-0/data,/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781864224577058-825-0/minicluster-data/ts-0/wal
uuid: "e3844982d2d44e3285b8408e1448d94e"
format_stamp: "Formatted at 2026-06-19 10:20:53 on dist-test-slave-6pt2"
server_key: "4ec0b34e3aaf5db811e35c0316eb2294"
server_key_iv: "c528b6932f62714a6a7297ccad8219e5"
server_key_version: "encryptionkey@0"
I20260619 10:20:53.925032 18903 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781864224577058-825-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781864224577058-825-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781864224577058-825-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 10:20:53.933754 18903 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 10:20:53.935511 18903 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 10:20:53.935667 18903 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 10:20:53.935880 18903 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 10:20:53.936178 18903 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 10:20:53.936231 18903 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 10:20:53.936264 18903 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 10:20:53.936291 18903 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 10:20:53.941769 18903 rpc_server.cc:307] RPC server started. Bound to: 127.0.206.65:38291
I20260619 10:20:53.941836 19031 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.206.65:38291 every 8 connection(s)
I20260619 10:20:53.942144 18903 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781864224577058-825-0/minicluster-data/ts-0/data/info.pb
Jun 19 10:20:53 dist-test-slave-6pt2 krb5kdc[18796](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864453, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.65@KRBTEST.COM for kudu/127.0.206.126@KRBTEST.COM
I20260619 10:20:53.950884 18900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:20:53.942929 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.65:32819 (local address 127.0.206.126:39125)
0619 10:20:53.942994 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0619 10:20:53.942996 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 10:20:53.943200 (+ 204us) server_negotiation.cc:407] Connection header received
0619 10:20:53.944060 (+ 860us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:20:53.944061 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:20:53.944070 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:20:53.944127 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:20:53.944665 (+ 538us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:20:53.945158 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:20:53.946100 (+ 942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:20:53.946297 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:20:53.946354 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:20:53.948352 (+ 1998us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 10:20:53.948354 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 10:20:53.948354 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 10:20:53.948369 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0619 10:20:53.949621 (+ 1252us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:20:53.950012 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:20:53.950013 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:20:53.950013 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:20:53.950076 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:20:53.950330 (+ 254us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:20:53.950331 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:20:53.950331 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:20:53.950437 (+ 106us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 10:20:53.950485 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0619 10:20:53.950750 (+ 265us) server_negotiation.cc:299] Negotiation successful
0619 10:20:53.950819 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260619 10:20:53.951104 19034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:20:53.942906 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:39125 (local address 127.0.206.65:32819)
0619 10:20:53.943069 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0619 10:20:53.943108 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0619 10:20:53.943978 (+ 870us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 10:20:53.944178 (+ 200us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 10:20:53.944181 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 10:20:53.944427 (+ 246us) client_negotiation.cc:189] Negotiated authn=SASL
0619 10:20:53.944597 (+ 170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:20:53.944601 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:20:53.945232 (+ 631us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:20:53.945232 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:20:53.946023 (+ 791us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:20:53.946025 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:20:53.946372 (+ 347us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:20:53.946373 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:20:53.946465 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:20:53.946920 (+ 455us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 10:20:53.946928 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 10:20:53.948271 (+ 1343us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 10:20:53.949713 (+ 1442us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 10:20:53.949720 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 10:20:53.949721 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 10:20:53.949936 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 10:20:53.950157 (+ 221us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 10:20:53.950158 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 10:20:53.950158 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 10:20:53.950262 (+ 104us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 10:20:53.950490 (+ 228us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 10:20:53.950494 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 10:20:53.950620 (+ 126us) client_negotiation.cc:769] Sending connection context
0619 10:20:53.950735 (+ 115us) client_negotiation.cc:240] Negotiation successful
0619 10:20:53.950860 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":93,"threads_started":1}
I20260619 10:20:53.951643 19032 heartbeater.cc:347] Connected to a master server at 127.0.206.126:39125
I20260619 10:20:53.951752 19032 heartbeater.cc:464] Registering TS with master...
I20260619 10:20:53.951929 19032 heartbeater.cc:511] Master 127.0.206.126:39125 requested a full tablet report, sending...
I20260619 10:20:53.952257 825 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTU9OP6/build/release/bin/kudu as pid 18903
I20260619 10:20:53.952363 825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781864224577058-825-0/minicluster-data/ts-0/wal/instance
I20260619 10:20:53.952524 18841 ts_manager.cc:194] Registered new tserver with Master: e3844982d2d44e3285b8408e1448d94e (127.0.206.65:38291)
I20260619 10:20:53.952577 825 external_mini_cluster.cc:1428] Setting key 64ea9964108577923bc976293cc108be
I20260619 10:20:53.953176 18841 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.206.65@KRBTEST.COM'} at 127.0.206.65:32819
WARNING: no policy specified for kudu/127.0.206.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.206.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.206.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781864224577058-825-0/krb5kdc/kudu_127.0.206.66.keytab.
Entry for principal kudu/127.0.206.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781864224577058-825-0/krb5kdc/kudu_127.0.206.66.keytab.
I20260619 10:20:53.975267 825 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTU9OP6/build/release/bin/kudu
/tmp/dist-test-taskTU9OP6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781864224577058-825-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781864224577058-825-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781864224577058-825-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781864224577058-825-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.206.66:0
--local_ip_for_outbound_sockets=127.0.206.66
--webserver_interface=127.0.206.66
--webserver_port=0
--tserver_master_addrs=127.0.206.126:39125
--keytab_file=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781864224577058-825-0/krb5kdc/kudu_127.0.206.66.keytab
--principal=kudu/127.0.206.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.206.84:44173
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781864224577058-825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781864224577058-825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781864224577058-825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 10:20:54.053612 19037 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 10:20:54.053804 19037 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 10:20:54.053828 19037 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 10:20:54.054891 19037 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 10:20:54.054957 19037 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 10:20:54.054981 19037 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.206.66
I20260619 10:20:54.056353 19037 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.206.84:44173
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781864224577058-825-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781864224577058-825-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781864224577058-825-0/krb5kdc/kudu_127.0.206.66.keytab
--principal=kudu/127.0.206.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.206.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781864224577058-825-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.206.66
--webserver_port=0
--tserver_master_addrs=127.0.206.126:39125
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19037
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.206.66
--log_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781864224577058-825-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 09:32:44 UTC on ccycloud.quasar-gbtldp.root.comops.site
build id 4455440
I20260619 10:20:54.056519 19037 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 10:20:54.056682 19037 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260619 10:20:54.061339 19037 server_base.cc:1034] running on GCE node
W20260619 10:20:54.061326 19042 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 10:20:54.061290 19045 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 10:20:54.061295 19043 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 10:20:54.061795 19037 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 10:20:54.062033 19037 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 10:20:54.063207 19037 hybrid_clock.cc:648] HybridClock initialized: now 1781864454063193 us; error 26 us; skew 500 ppm
Jun 19 10:20:54 dist-test-slave-6pt2 krb5kdc[18796](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864454, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 10:20:54.065510 19037 init.cc:377] Logged in from keytab as kudu/127.0.206.66@KRBTEST.COM (short username kudu)
I20260619 10:20:54.065901 19037 webserver.cc:460] Webserver started at http://127.0.206.66:41807/ using document root <none> and password file <none>
I20260619 10:20:54.066118 19037 fs_manager.cc:359] Metadata directory not provided
I20260619 10:20:54.066185 19037 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 10:20:54.066331 19037 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 10:20:54.067370 19037 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781864224577058-825-0/minicluster-data/ts-1/data/instance:
uuid: "720e2ee687d34cdc8425f0d12991b9d2"
format_stamp: "Formatted at 2026-06-19 10:20:54 on dist-test-slave-6pt2"
server_key: "e593f9b55da0f3ffed6ba970efacd2f1"
server_key_iv: "9a1daddf4e72e8d557f3e6709ba319d6"
server_key_version: "encryptionkey@0"
I20260619 10:20:54.067714 19037 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781864224577058-825-0/minicluster-data/ts-1/wal/instance:
uuid: "720e2ee687d34cdc8425f0d12991b9d2"
format_stamp: "Formatted at 2026-06-19 10:20:54 on dist-test-slave-6pt2"
server_key: "e593f9b55da0f3ffed6ba970efacd2f1"
server_key_iv: "9a1daddf4e72e8d557f3e6709ba319d6"
server_key_version: "encryptionkey@0"
I20260619 10:20:54.069008 19037 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260619 10:20:54.069698 19052 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 10:20:54.069880 19037 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260619 10:20:54.069981 19037 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781864224577058-825-0/minicluster-data/ts-1/data,/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781864224577058-825-0/minicluster-data/ts-1/wal
uuid: "720e2ee687d34cdc8425f0d12991b9d2"
format_stamp: "Formatted at 2026-06-19 10:20:54 on dist-test-slave-6pt2"
server_key: "e593f9b55da0f3ffed6ba970efacd2f1"
server_key_iv: "9a1daddf4e72e8d557f3e6709ba319d6"
server_key_version: "encryptionkey@0"
I20260619 10:20:54.070055 19037 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781864224577058-825-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781864224577058-825-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781864224577058-825-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 10:20:54.081458 19037 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 10:20:54.083244 19037 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 10:20:54.083403 19037 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 10:20:54.083611 19037 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 10:20:54.083894 19037 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 10:20:54.083956 19037 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 10:20:54.083995 19037 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 10:20:54.084023 19037 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 10:20:54.089426 19037 rpc_server.cc:307] RPC server started. Bound to: 127.0.206.66:41331
I20260619 10:20:54.089499 19165 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.206.66:41331 every 8 connection(s)
I20260619 10:20:54.089798 19037 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781864224577058-825-0/minicluster-data/ts-1/data/info.pb
Jun 19 10:20:54 dist-test-slave-6pt2 krb5kdc[18796](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864454, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.66@KRBTEST.COM for kudu/127.0.206.126@KRBTEST.COM
I20260619 10:20:54.098997 18900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:20:54.090524 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.66:47463 (local address 127.0.206.126:39125)
0619 10:20:54.090583 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0619 10:20:54.090585 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 10:20:54.090778 (+ 193us) server_negotiation.cc:407] Connection header received
0619 10:20:54.091689 (+ 911us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:20:54.091691 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:20:54.091701 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:20:54.091766 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:20:54.092346 (+ 580us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:20:54.092838 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:20:54.093764 (+ 926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:20:54.093948 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:20:54.094006 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:20:54.096259 (+ 2253us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 10:20:54.096261 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 10:20:54.096261 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 10:20:54.096269 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 10:20:54.097568 (+ 1299us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:20:54.097993 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:20:54.097994 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:20:54.097994 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:20:54.098051 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:20:54.098294 (+ 243us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:20:54.098295 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:20:54.098295 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:20:54.098414 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 10:20:54.098492 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0619 10:20:54.098824 (+ 332us) server_negotiation.cc:299] Negotiation successful
0619 10:20:54.098909 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260619 10:20:54.099177 19168 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:20:54.090485 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:39125 (local address 127.0.206.66:47463)
0619 10:20:54.090656 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0619 10:20:54.090696 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0619 10:20:54.091569 (+ 873us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 10:20:54.091836 (+ 267us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 10:20:54.091839 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 10:20:54.092086 (+ 247us) client_negotiation.cc:189] Negotiated authn=SASL
0619 10:20:54.092263 (+ 177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:20:54.092267 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:20:54.092916 (+ 649us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:20:54.092916 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:20:54.093693 (+ 777us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:20:54.093695 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:20:54.094025 (+ 330us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:20:54.094025 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:20:54.094179 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:20:54.094775 (+ 596us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 10:20:54.094783 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 10:20:54.096163 (+ 1380us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 10:20:54.097667 (+ 1504us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 10:20:54.097673 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 10:20:54.097675 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 10:20:54.097901 (+ 226us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 10:20:54.098118 (+ 217us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 10:20:54.098119 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 10:20:54.098119 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 10:20:54.098227 (+ 108us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 10:20:54.098482 (+ 255us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 10:20:54.098487 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 10:20:54.098692 (+ 205us) client_negotiation.cc:769] Sending connection context
0619 10:20:54.098804 (+ 112us) client_negotiation.cc:240] Negotiation successful
0619 10:20:54.098961 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":81,"threads_started":1}
I20260619 10:20:54.099413 825 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTU9OP6/build/release/bin/kudu as pid 19037
I20260619 10:20:54.099531 825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781864224577058-825-0/minicluster-data/ts-1/wal/instance
I20260619 10:20:54.099737 825 external_mini_cluster.cc:1428] Setting key cfb9d39f778ad9d5c741835ac586f8db
I20260619 10:20:54.099735 19166 heartbeater.cc:347] Connected to a master server at 127.0.206.126:39125
I20260619 10:20:54.099817 19166 heartbeater.cc:464] Registering TS with master...
I20260619 10:20:54.100013 19166 heartbeater.cc:511] Master 127.0.206.126:39125 requested a full tablet report, sending...
I20260619 10:20:54.100361 18841 ts_manager.cc:194] Registered new tserver with Master: 720e2ee687d34cdc8425f0d12991b9d2 (127.0.206.66:41331)
I20260619 10:20:54.100831 18841 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.206.66@KRBTEST.COM'} at 127.0.206.66:47463
WARNING: no policy specified for kudu/127.0.206.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.206.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.206.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781864224577058-825-0/krb5kdc/kudu_127.0.206.67.keytab.
Entry for principal kudu/127.0.206.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781864224577058-825-0/krb5kdc/kudu_127.0.206.67.keytab.
I20260619 10:20:54.122766 825 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTU9OP6/build/release/bin/kudu
/tmp/dist-test-taskTU9OP6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781864224577058-825-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781864224577058-825-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781864224577058-825-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781864224577058-825-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.206.67:0
--local_ip_for_outbound_sockets=127.0.206.67
--webserver_interface=127.0.206.67
--webserver_port=0
--tserver_master_addrs=127.0.206.126:39125
--keytab_file=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781864224577058-825-0/krb5kdc/kudu_127.0.206.67.keytab
--principal=kudu/127.0.206.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.206.84:44173
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781864224577058-825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781864224577058-825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781864224577058-825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 10:20:54.201859 19171 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 10:20:54.202060 19171 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 10:20:54.202081 19171 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 10:20:54.203217 19171 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 10:20:54.203261 19171 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 10:20:54.203287 19171 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.206.67
I20260619 10:20:54.204651 19171 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.206.84:44173
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781864224577058-825-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781864224577058-825-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781864224577058-825-0/krb5kdc/kudu_127.0.206.67.keytab
--principal=kudu/127.0.206.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.206.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781864224577058-825-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.206.67
--webserver_port=0
--tserver_master_addrs=127.0.206.126:39125
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19171
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.206.67
--log_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781864224577058-825-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 09:32:44 UTC on ccycloud.quasar-gbtldp.root.comops.site
build id 4455440
I20260619 10:20:54.204860 19171 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 10:20:54.205031 19171 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 10:20:54.209656 19179 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 10:20:54.209674 19177 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 10:20:54.209725 19176 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 10:20:54.209748 19171 server_base.cc:1034] running on GCE node
I20260619 10:20:54.210062 19171 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 10:20:54.210319 19171 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 10:20:54.211458 19171 hybrid_clock.cc:648] HybridClock initialized: now 1781864454211453 us; error 31 us; skew 500 ppm
Jun 19 10:20:54 dist-test-slave-6pt2 krb5kdc[18796](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864454, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 10:20:54.213790 19171 init.cc:377] Logged in from keytab as kudu/127.0.206.67@KRBTEST.COM (short username kudu)
I20260619 10:20:54.214210 19171 webserver.cc:460] Webserver started at http://127.0.206.67:40743/ using document root <none> and password file <none>
I20260619 10:20:54.214429 19171 fs_manager.cc:359] Metadata directory not provided
I20260619 10:20:54.214496 19171 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 10:20:54.214663 19171 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 10:20:54.215682 19171 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781864224577058-825-0/minicluster-data/ts-2/data/instance:
uuid: "1d5504d3acf74ce393bad25f554af3c6"
format_stamp: "Formatted at 2026-06-19 10:20:54 on dist-test-slave-6pt2"
server_key: "88f87c9431aa2b1d399dbb8edd396158"
server_key_iv: "0a08bafe44fd5c46cea585143ccbd928"
server_key_version: "encryptionkey@0"
I20260619 10:20:54.216029 19171 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781864224577058-825-0/minicluster-data/ts-2/wal/instance:
uuid: "1d5504d3acf74ce393bad25f554af3c6"
format_stamp: "Formatted at 2026-06-19 10:20:54 on dist-test-slave-6pt2"
server_key: "88f87c9431aa2b1d399dbb8edd396158"
server_key_iv: "0a08bafe44fd5c46cea585143ccbd928"
server_key_version: "encryptionkey@0"
I20260619 10:20:54.217374 19171 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 10:20:54.218108 19186 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 10:20:54.218277 19171 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 10:20:54.218374 19171 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781864224577058-825-0/minicluster-data/ts-2/data,/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781864224577058-825-0/minicluster-data/ts-2/wal
uuid: "1d5504d3acf74ce393bad25f554af3c6"
format_stamp: "Formatted at 2026-06-19 10:20:54 on dist-test-slave-6pt2"
server_key: "88f87c9431aa2b1d399dbb8edd396158"
server_key_iv: "0a08bafe44fd5c46cea585143ccbd928"
server_key_version: "encryptionkey@0"
I20260619 10:20:54.218458 19171 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781864224577058-825-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781864224577058-825-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781864224577058-825-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 10:20:54.230999 19171 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 10:20:54.232628 19171 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 10:20:54.232776 19171 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 10:20:54.232976 19171 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 10:20:54.233248 19171 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 10:20:54.233309 19171 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 10:20:54.233350 19171 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 10:20:54.233373 19171 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 10:20:54.239374 19171 rpc_server.cc:307] RPC server started. Bound to: 127.0.206.67:41153
I20260619 10:20:54.239424 19299 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.206.67:41153 every 8 connection(s)
I20260619 10:20:54.239701 19171 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781864224577058-825-0/minicluster-data/ts-2/data/info.pb
Jun 19 10:20:54 dist-test-slave-6pt2 krb5kdc[18796](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864454, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.67@KRBTEST.COM for kudu/127.0.206.126@KRBTEST.COM
I20260619 10:20:54.247205 825 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTU9OP6/build/release/bin/kudu as pid 19171
I20260619 10:20:54.247341 825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781864224577058-825-0/minicluster-data/ts-2/wal/instance
I20260619 10:20:54.247542 825 external_mini_cluster.cc:1428] Setting key a2d256be1b80013713b791a4f7134b72
I20260619 10:20:54.249215 18900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:20:54.240384 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.67:47111 (local address 127.0.206.126:39125)
0619 10:20:54.240437 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0619 10:20:54.240439 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 10:20:54.240618 (+ 179us) server_negotiation.cc:407] Connection header received
0619 10:20:54.241509 (+ 891us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:20:54.241510 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:20:54.241522 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:20:54.241575 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:20:54.242158 (+ 583us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:20:54.242735 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:20:54.243710 (+ 975us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:20:54.243921 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:20:54.244007 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:20:54.246024 (+ 2017us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 10:20:54.246025 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 10:20:54.246025 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 10:20:54.246031 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 10:20:54.247706 (+ 1675us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:20:54.248214 (+ 508us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:20:54.248215 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:20:54.248215 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:20:54.248275 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:20:54.248575 (+ 300us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:20:54.248575 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:20:54.248576 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:20:54.248691 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 10:20:54.248787 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0619 10:20:54.249046 (+ 259us) server_negotiation.cc:299] Negotiation successful
0619 10:20:54.249135 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260619 10:20:54.249439 19302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:20:54.240369 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:39125 (local address 127.0.206.67:47111)
0619 10:20:54.240497 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0619 10:20:54.240536 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0619 10:20:54.241399 (+ 863us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 10:20:54.241638 (+ 239us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 10:20:54.241641 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 10:20:54.241886 (+ 245us) client_negotiation.cc:189] Negotiated authn=SASL
0619 10:20:54.242061 (+ 175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:20:54.242065 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:20:54.242833 (+ 768us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:20:54.242834 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:20:54.243618 (+ 784us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:20:54.243620 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:20:54.244015 (+ 395us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:20:54.244016 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:20:54.244134 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:20:54.244550 (+ 416us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 10:20:54.244557 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0619 10:20:54.245942 (+ 1385us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 10:20:54.247787 (+ 1845us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 10:20:54.247793 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 10:20:54.247795 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 10:20:54.248127 (+ 332us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 10:20:54.248359 (+ 232us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 10:20:54.248360 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 10:20:54.248360 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 10:20:54.248485 (+ 125us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 10:20:54.248777 (+ 292us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 10:20:54.248782 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 10:20:54.248919 (+ 137us) client_negotiation.cc:769] Sending connection context
0619 10:20:54.249001 (+ 82us) client_negotiation.cc:240] Negotiation successful
0619 10:20:54.249206 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":94,"thread_start_us":61,"threads_started":1}
I20260619 10:20:54.250013 19300 heartbeater.cc:347] Connected to a master server at 127.0.206.126:39125
I20260619 10:20:54.250105 19300 heartbeater.cc:464] Registering TS with master...
I20260619 10:20:54.250274 19300 heartbeater.cc:511] Master 127.0.206.126:39125 requested a full tablet report, sending...
I20260619 10:20:54.250633 18841 ts_manager.cc:194] Registered new tserver with Master: 1d5504d3acf74ce393bad25f554af3c6 (127.0.206.67:41153)
I20260619 10:20:54.250959 18841 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.206.67@KRBTEST.COM'} at 127.0.206.67:47111
I20260619 10:20:54.259344 825 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 10:20:54.265974 18900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:20:54.260625 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58446 (local address 127.0.206.126:39125)
0619 10:20:54.260688 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0619 10:20:54.260690 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 10:20:54.260790 (+ 100us) server_negotiation.cc:407] Connection header received
0619 10:20:54.260814 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:20:54.260815 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:20:54.260823 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:20:54.260872 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:20:54.261356 (+ 484us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:20:54.261806 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:20:54.262475 (+ 669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:20:54.262704 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:20:54.262753 (+ 49us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:20:54.263852 (+ 1099us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 10:20:54.263853 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 10:20:54.263854 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 10:20:54.263859 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0619 10:20:54.264974 (+ 1115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:20:54.265334 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:20:54.265334 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:20:54.265335 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:20:54.265382 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:20:54.265524 (+ 142us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:20:54.265524 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:20:54.265524 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:20:54.265634 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 10:20:54.265708 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0619 10:20:54.265815 (+ 107us) server_negotiation.cc:299] Negotiation successful
0619 10:20:54.265860 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260619 10:20:54.267611 18841 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58446:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 10:20:54.269191 18841 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 10:20:54.275463 19312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:20:54.271199 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.66:41331 (local address 127.0.0.1:39460)
0619 10:20:54.271582 (+ 383us) negotiation.cc:107] Waiting for socket to connect
0619 10:20:54.271590 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0619 10:20:54.271636 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 10:20:54.272037 (+ 401us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 10:20:54.272037 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 10:20:54.272039 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 10:20:54.272075 (+ 36us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:20:54.272076 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:20:54.273139 (+ 1063us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:20:54.273140 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:20:54.274443 (+ 1303us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:20:54.274445 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:20:54.275176 (+ 731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:20:54.275176 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:20:54.275253 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:20:54.275265 (+ 12us) client_negotiation.cc:769] Sending connection context
0619 10:20:54.275336 (+ 71us) client_negotiation.cc:240] Negotiation successful
0619 10:20:54.275391 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":369,"mutex_wait_us":3,"thread_start_us":67,"threads_started":1}
I20260619 10:20:54.275476 19311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:20:54.271351 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39460 (local address 127.0.206.66:41331)
0619 10:20:54.271538 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0619 10:20:54.271544 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 10:20:54.271707 (+ 163us) server_negotiation.cc:407] Connection header received
0619 10:20:54.271745 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:20:54.271747 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:20:54.271762 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:20:54.271875 (+ 113us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 10:20:54.272130 (+ 255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:20:54.273033 (+ 903us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:20:54.274523 (+ 1490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:20:54.275089 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:20:54.275175 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:20:54.275232 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0619 10:20:54.275357 (+ 125us) server_negotiation.cc:299] Negotiation successful
0619 10:20:54.275394 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":158,"thread_start_us":69,"threads_started":1}
I20260619 10:20:54.275761 19308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:20:54.271414 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.67:41153 (local address 127.0.0.1:50602)
0619 10:20:54.271974 (+ 560us) negotiation.cc:107] Waiting for socket to connect
0619 10:20:54.271984 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 10:20:54.272037 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 10:20:54.272434 (+ 397us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 10:20:54.272435 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 10:20:54.272437 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 10:20:54.272473 (+ 36us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:20:54.272475 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:20:54.273601 (+ 1126us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:20:54.273602 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:20:54.274565 (+ 963us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:20:54.274568 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:20:54.275532 (+ 964us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:20:54.275533 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:20:54.275589 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:20:54.275593 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 10:20:54.275659 (+ 66us) client_negotiation.cc:240] Negotiation successful
0619 10:20:54.275694 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":546,"thread_start_us":44,"threads_started":1}
I20260619 10:20:54.275805 19313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:20:54.271514 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50602 (local address 127.0.206.67:41153)
0619 10:20:54.271669 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0619 10:20:54.271674 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 10:20:54.272215 (+ 541us) server_negotiation.cc:407] Connection header received
0619 10:20:54.272258 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:20:54.272260 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:20:54.272276 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:20:54.272346 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 10:20:54.272564 (+ 218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:20:54.273503 (+ 939us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:20:54.274679 (+ 1176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:20:54.275382 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:20:54.275445 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:20:54.275495 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0619 10:20:54.275678 (+ 183us) server_negotiation.cc:299] Negotiation successful
0619 10:20:54.275728 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":122,"thread_start_us":89,"threads_started":1}
I20260619 10:20:54.276275 19100 tablet_service.cc:1467] Processing CreateTablet for tablet 70935534710b45dd91d2c32320085806 (DEFAULT_TABLE table=test-table [id=2264b0edb04f4e7d9aedd905115f7d47]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 10:20:54.276665 19100 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 70935534710b45dd91d2c32320085806. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 10:20:54.276695 19234 tablet_service.cc:1467] Processing CreateTablet for tablet 70935534710b45dd91d2c32320085806 (DEFAULT_TABLE table=test-table [id=2264b0edb04f4e7d9aedd905115f7d47]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 10:20:54.277009 19234 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 70935534710b45dd91d2c32320085806. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 10:20:54.278735 19309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:20:54.271194 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.65:38291 (local address 127.0.0.1:57618)
0619 10:20:54.271520 (+ 326us) negotiation.cc:107] Waiting for socket to connect
0619 10:20:54.271559 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0619 10:20:54.271624 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 10:20:54.271871 (+ 247us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 10:20:54.271874 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 10:20:54.271875 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 10:20:54.271948 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:20:54.271950 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:20:54.273304 (+ 1354us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:20:54.273304 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:20:54.274437 (+ 1133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:20:54.274440 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:20:54.275715 (+ 1275us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:20:54.275716 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:20:54.275795 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:20:54.275799 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 10:20:54.275855 (+ 56us) client_negotiation.cc:240] Negotiation successful
0619 10:20:54.278653 (+ 2798us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":299,"mutex_wait_us":2,"thread_start_us":68,"threads_started":1}
I20260619 10:20:54.278825 19310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:20:54.271306 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57618 (local address 127.0.206.65:38291)
0619 10:20:54.271708 (+ 402us) server_negotiation.cc:206] Beginning negotiation
0619 10:20:54.271711 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 10:20:54.271720 (+ 9us) server_negotiation.cc:407] Connection header received
0619 10:20:54.271763 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:20:54.271764 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:20:54.271776 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:20:54.271843 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 10:20:54.272066 (+ 223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:20:54.273194 (+ 1128us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:20:54.274679 (+ 1485us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:20:54.275373 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:20:54.275446 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:20:54.275494 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0619 10:20:54.278692 (+ 3198us) server_negotiation.cc:299] Negotiation successful
0619 10:20:54.278739 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":370,"thread_start_us":113,"threads_started":1}
I20260619 10:20:54.279616 18966 tablet_service.cc:1467] Processing CreateTablet for tablet 70935534710b45dd91d2c32320085806 (DEFAULT_TABLE table=test-table [id=2264b0edb04f4e7d9aedd905115f7d47]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 10:20:54.279734 19315 tablet_bootstrap.cc:492] T 70935534710b45dd91d2c32320085806 P 1d5504d3acf74ce393bad25f554af3c6: Bootstrap starting.
I20260619 10:20:54.279734 19314 tablet_bootstrap.cc:492] T 70935534710b45dd91d2c32320085806 P 720e2ee687d34cdc8425f0d12991b9d2: Bootstrap starting.
I20260619 10:20:54.279976 18966 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 70935534710b45dd91d2c32320085806. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 10:20:54.280406 19315 tablet_bootstrap.cc:654] T 70935534710b45dd91d2c32320085806 P 1d5504d3acf74ce393bad25f554af3c6: Neither blocks nor log segments found. Creating new log.
I20260619 10:20:54.280406 19314 tablet_bootstrap.cc:654] T 70935534710b45dd91d2c32320085806 P 720e2ee687d34cdc8425f0d12991b9d2: Neither blocks nor log segments found. Creating new log.
I20260619 10:20:54.280983 19314 log.cc:824] T 70935534710b45dd91d2c32320085806 P 720e2ee687d34cdc8425f0d12991b9d2: Log is configured to *not* fsync() on all Append() calls
I20260619 10:20:54.280983 19315 log.cc:824] T 70935534710b45dd91d2c32320085806 P 1d5504d3acf74ce393bad25f554af3c6: Log is configured to *not* fsync() on all Append() calls
I20260619 10:20:54.281683 19314 tablet_bootstrap.cc:492] T 70935534710b45dd91d2c32320085806 P 720e2ee687d34cdc8425f0d12991b9d2: No bootstrap required, opened a new log
I20260619 10:20:54.281769 19314 ts_tablet_manager.cc:1397] T 70935534710b45dd91d2c32320085806 P 720e2ee687d34cdc8425f0d12991b9d2: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260619 10:20:54.281935 19315 tablet_bootstrap.cc:492] T 70935534710b45dd91d2c32320085806 P 1d5504d3acf74ce393bad25f554af3c6: No bootstrap required, opened a new log
I20260619 10:20:54.282007 19315 ts_tablet_manager.cc:1397] T 70935534710b45dd91d2c32320085806 P 1d5504d3acf74ce393bad25f554af3c6: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260619 10:20:54.282703 19318 tablet_bootstrap.cc:492] T 70935534710b45dd91d2c32320085806 P e3844982d2d44e3285b8408e1448d94e: Bootstrap starting.
I20260619 10:20:54.283322 19318 tablet_bootstrap.cc:654] T 70935534710b45dd91d2c32320085806 P e3844982d2d44e3285b8408e1448d94e: Neither blocks nor log segments found. Creating new log.
I20260619 10:20:54.283782 19318 log.cc:824] T 70935534710b45dd91d2c32320085806 P e3844982d2d44e3285b8408e1448d94e: Log is configured to *not* fsync() on all Append() calls
I20260619 10:20:54.283890 19314 raft_consensus.cc:348] T 70935534710b45dd91d2c32320085806 P 720e2ee687d34cdc8425f0d12991b9d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3844982d2d44e3285b8408e1448d94e" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 38291 } } peers { permanent_uuid: "720e2ee687d34cdc8425f0d12991b9d2" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 41331 } } peers { permanent_uuid: "1d5504d3acf74ce393bad25f554af3c6" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 41153 } }
I20260619 10:20:54.284085 19314 raft_consensus.cc:374] T 70935534710b45dd91d2c32320085806 P 720e2ee687d34cdc8425f0d12991b9d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 10:20:54.284154 19314 raft_consensus.cc:729] T 70935534710b45dd91d2c32320085806 P 720e2ee687d34cdc8425f0d12991b9d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 720e2ee687d34cdc8425f0d12991b9d2, State: Initialized, Role: FOLLOWER
I20260619 10:20:54.284318 19314 consensus_queue.cc:260] T 70935534710b45dd91d2c32320085806 P 720e2ee687d34cdc8425f0d12991b9d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3844982d2d44e3285b8408e1448d94e" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 38291 } } peers { permanent_uuid: "720e2ee687d34cdc8425f0d12991b9d2" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 41331 } } peers { permanent_uuid: "1d5504d3acf74ce393bad25f554af3c6" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 41153 } }
I20260619 10:20:54.284480 19318 tablet_bootstrap.cc:492] T 70935534710b45dd91d2c32320085806 P e3844982d2d44e3285b8408e1448d94e: No bootstrap required, opened a new log
I20260619 10:20:54.284544 19318 ts_tablet_manager.cc:1397] T 70935534710b45dd91d2c32320085806 P e3844982d2d44e3285b8408e1448d94e: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260619 10:20:54.284583 19314 ts_tablet_manager.cc:1428] T 70935534710b45dd91d2c32320085806 P 720e2ee687d34cdc8425f0d12991b9d2: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260619 10:20:54.284546 19315 raft_consensus.cc:348] T 70935534710b45dd91d2c32320085806 P 1d5504d3acf74ce393bad25f554af3c6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3844982d2d44e3285b8408e1448d94e" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 38291 } } peers { permanent_uuid: "720e2ee687d34cdc8425f0d12991b9d2" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 41331 } } peers { permanent_uuid: "1d5504d3acf74ce393bad25f554af3c6" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 41153 } }
I20260619 10:20:54.284703 19166 heartbeater.cc:503] Master 127.0.206.126:39125 was elected leader, sending a full tablet report...
I20260619 10:20:54.284747 19315 raft_consensus.cc:374] T 70935534710b45dd91d2c32320085806 P 1d5504d3acf74ce393bad25f554af3c6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 10:20:54.284785 19315 raft_consensus.cc:729] T 70935534710b45dd91d2c32320085806 P 1d5504d3acf74ce393bad25f554af3c6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1d5504d3acf74ce393bad25f554af3c6, State: Initialized, Role: FOLLOWER
I20260619 10:20:54.284909 19315 consensus_queue.cc:260] T 70935534710b45dd91d2c32320085806 P 1d5504d3acf74ce393bad25f554af3c6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3844982d2d44e3285b8408e1448d94e" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 38291 } } peers { permanent_uuid: "720e2ee687d34cdc8425f0d12991b9d2" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 41331 } } peers { permanent_uuid: "1d5504d3acf74ce393bad25f554af3c6" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 41153 } }
I20260619 10:20:54.285158 19315 ts_tablet_manager.cc:1428] T 70935534710b45dd91d2c32320085806 P 1d5504d3acf74ce393bad25f554af3c6: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260619 10:20:54.285271 19300 heartbeater.cc:503] Master 127.0.206.126:39125 was elected leader, sending a full tablet report...
I20260619 10:20:54.286535 19318 raft_consensus.cc:348] T 70935534710b45dd91d2c32320085806 P e3844982d2d44e3285b8408e1448d94e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3844982d2d44e3285b8408e1448d94e" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 38291 } } peers { permanent_uuid: "720e2ee687d34cdc8425f0d12991b9d2" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 41331 } } peers { permanent_uuid: "1d5504d3acf74ce393bad25f554af3c6" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 41153 } }
I20260619 10:20:54.286751 19318 raft_consensus.cc:374] T 70935534710b45dd91d2c32320085806 P e3844982d2d44e3285b8408e1448d94e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 10:20:54.286818 19318 raft_consensus.cc:729] T 70935534710b45dd91d2c32320085806 P e3844982d2d44e3285b8408e1448d94e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e3844982d2d44e3285b8408e1448d94e, State: Initialized, Role: FOLLOWER
I20260619 10:20:54.286965 19318 consensus_queue.cc:260] T 70935534710b45dd91d2c32320085806 P e3844982d2d44e3285b8408e1448d94e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3844982d2d44e3285b8408e1448d94e" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 38291 } } peers { permanent_uuid: "720e2ee687d34cdc8425f0d12991b9d2" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 41331 } } peers { permanent_uuid: "1d5504d3acf74ce393bad25f554af3c6" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 41153 } }
I20260619 10:20:54.287288 19318 ts_tablet_manager.cc:1428] T 70935534710b45dd91d2c32320085806 P e3844982d2d44e3285b8408e1448d94e: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20260619 10:20:54.287405 19032 heartbeater.cc:503] Master 127.0.206.126:39125 was elected leader, sending a full tablet report...
W20260619 10:20:54.340344 19167 tablet.cc:2369] T 70935534710b45dd91d2c32320085806 P 720e2ee687d34cdc8425f0d12991b9d2: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260619 10:20:54.442893 19033 tablet.cc:2369] T 70935534710b45dd91d2c32320085806 P e3844982d2d44e3285b8408e1448d94e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 10:20:54.445932 19320 raft_consensus.cc:482] T 70935534710b45dd91d2c32320085806 P 720e2ee687d34cdc8425f0d12991b9d2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 10:20:54.446028 19320 raft_consensus.cc:504] T 70935534710b45dd91d2c32320085806 P 720e2ee687d34cdc8425f0d12991b9d2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3844982d2d44e3285b8408e1448d94e" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 38291 } } peers { permanent_uuid: "720e2ee687d34cdc8425f0d12991b9d2" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 41331 } } peers { permanent_uuid: "1d5504d3acf74ce393bad25f554af3c6" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 41153 } }
I20260619 10:20:54.446461 19320 leader_election.cc:288] T 70935534710b45dd91d2c32320085806 P 720e2ee687d34cdc8425f0d12991b9d2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e3844982d2d44e3285b8408e1448d94e (127.0.206.65:38291), 1d5504d3acf74ce393bad25f554af3c6 (127.0.206.67:41153)
I20260619 10:20:54.449673 19313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:20:54.446973 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.66:44825 (local address 127.0.206.67:41153)
0619 10:20:54.447045 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0619 10:20:54.447048 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 10:20:54.447148 (+ 100us) server_negotiation.cc:407] Connection header received
0619 10:20:54.447217 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:20:54.447218 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:20:54.447229 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:20:54.447300 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 10:20:54.447467 (+ 167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:20:54.447927 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:20:54.448880 (+ 953us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:20:54.449294 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:20:54.449387 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:20:54.449396 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0619 10:20:54.449551 (+ 155us) server_negotiation.cc:299] Negotiation successful
0619 10:20:54.449583 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260619 10:20:54.449692 19310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:20:54.446814 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.66:34611 (local address 127.0.206.65:38291)
0619 10:20:54.446882 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0619 10:20:54.446884 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 10:20:54.446896 (+ 12us) server_negotiation.cc:407] Connection header received
0619 10:20:54.446968 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:20:54.446969 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:20:54.446981 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:20:54.447047 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 10:20:54.447235 (+ 188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:20:54.447704 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:20:54.448848 (+ 1144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:20:54.449296 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:20:54.449393 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:20:54.449401 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0619 10:20:54.449559 (+ 158us) server_negotiation.cc:299] Negotiation successful
0619 10:20:54.449613 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260619 10:20:54.449692 19168 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:20:54.446751 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.65:38291 (local address 127.0.206.66:34611)
0619 10:20:54.446821 (+ 70us) negotiation.cc:107] Waiting for socket to connect
0619 10:20:54.446834 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 10:20:54.446904 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 10:20:54.447101 (+ 197us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 10:20:54.447103 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 10:20:54.447104 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 10:20:54.447161 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:20:54.447163 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:20:54.447819 (+ 656us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:20:54.447820 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:20:54.448729 (+ 909us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:20:54.448732 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:20:54.449389 (+ 657us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:20:54.449390 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:20:54.449468 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:20:54.449477 (+ 9us) client_negotiation.cc:769] Sending connection context
0619 10:20:54.449550 (+ 73us) client_negotiation.cc:240] Negotiation successful
0619 10:20:54.449591 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":47}
I20260619 10:20:54.449697 19323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:20:54.446886 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.67:41153 (local address 127.0.206.66:44825)
0619 10:20:54.447068 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0619 10:20:54.447077 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0619 10:20:54.447160 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 10:20:54.447304 (+ 144us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 10:20:54.447305 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 10:20:54.447306 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 10:20:54.447383 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:20:54.447385 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:20:54.448006 (+ 621us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:20:54.448007 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:20:54.448796 (+ 789us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:20:54.448799 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:20:54.449385 (+ 586us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:20:54.449386 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:20:54.449476 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:20:54.449478 (+ 2us) client_negotiation.cc:769] Sending connection context
0619 10:20:54.449546 (+ 68us) client_negotiation.cc:240] Negotiation successful
0619 10:20:54.449583 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":165,"mutex_wait_us":11,"thread_start_us":104,"threads_started":1}
I20260619 10:20:54.450120 19254 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "70935534710b45dd91d2c32320085806" candidate_uuid: "720e2ee687d34cdc8425f0d12991b9d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1d5504d3acf74ce393bad25f554af3c6" is_pre_election: true
I20260619 10:20:54.450117 18986 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "70935534710b45dd91d2c32320085806" candidate_uuid: "720e2ee687d34cdc8425f0d12991b9d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e3844982d2d44e3285b8408e1448d94e" is_pre_election: true
I20260619 10:20:54.450263 18986 raft_consensus.cc:2454] T 70935534710b45dd91d2c32320085806 P e3844982d2d44e3285b8408e1448d94e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 720e2ee687d34cdc8425f0d12991b9d2 in term 0.
I20260619 10:20:54.450263 19254 raft_consensus.cc:2454] T 70935534710b45dd91d2c32320085806 P 1d5504d3acf74ce393bad25f554af3c6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 720e2ee687d34cdc8425f0d12991b9d2 in term 0.
I20260619 10:20:54.450467 19056 leader_election.cc:302] T 70935534710b45dd91d2c32320085806 P 720e2ee687d34cdc8425f0d12991b9d2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 720e2ee687d34cdc8425f0d12991b9d2, e3844982d2d44e3285b8408e1448d94e; no voters:
I20260619 10:20:54.450613 19320 raft_consensus.cc:2781] T 70935534710b45dd91d2c32320085806 P 720e2ee687d34cdc8425f0d12991b9d2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 10:20:54.450665 19320 raft_consensus.cc:482] T 70935534710b45dd91d2c32320085806 P 720e2ee687d34cdc8425f0d12991b9d2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 10:20:54.450707 19320 raft_consensus.cc:3037] T 70935534710b45dd91d2c32320085806 P 720e2ee687d34cdc8425f0d12991b9d2 [term 0 FOLLOWER]: Advancing to term 1
I20260619 10:20:54.451675 19320 raft_consensus.cc:504] T 70935534710b45dd91d2c32320085806 P 720e2ee687d34cdc8425f0d12991b9d2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3844982d2d44e3285b8408e1448d94e" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 38291 } } peers { permanent_uuid: "720e2ee687d34cdc8425f0d12991b9d2" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 41331 } } peers { permanent_uuid: "1d5504d3acf74ce393bad25f554af3c6" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 41153 } }
I20260619 10:20:54.451808 19320 leader_election.cc:288] T 70935534710b45dd91d2c32320085806 P 720e2ee687d34cdc8425f0d12991b9d2 [CANDIDATE]: Term 1 election: Requested vote from peers e3844982d2d44e3285b8408e1448d94e (127.0.206.65:38291), 1d5504d3acf74ce393bad25f554af3c6 (127.0.206.67:41153)
I20260619 10:20:54.451937 18986 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "70935534710b45dd91d2c32320085806" candidate_uuid: "720e2ee687d34cdc8425f0d12991b9d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e3844982d2d44e3285b8408e1448d94e"
I20260619 10:20:54.451967 19254 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "70935534710b45dd91d2c32320085806" candidate_uuid: "720e2ee687d34cdc8425f0d12991b9d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1d5504d3acf74ce393bad25f554af3c6"
I20260619 10:20:54.452019 18986 raft_consensus.cc:3037] T 70935534710b45dd91d2c32320085806 P e3844982d2d44e3285b8408e1448d94e [term 0 FOLLOWER]: Advancing to term 1
I20260619 10:20:54.452030 19254 raft_consensus.cc:3037] T 70935534710b45dd91d2c32320085806 P 1d5504d3acf74ce393bad25f554af3c6 [term 0 FOLLOWER]: Advancing to term 1
I20260619 10:20:54.452859 19254 raft_consensus.cc:2454] T 70935534710b45dd91d2c32320085806 P 1d5504d3acf74ce393bad25f554af3c6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 720e2ee687d34cdc8425f0d12991b9d2 in term 1.
I20260619 10:20:54.452859 18986 raft_consensus.cc:2454] T 70935534710b45dd91d2c32320085806 P e3844982d2d44e3285b8408e1448d94e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 720e2ee687d34cdc8425f0d12991b9d2 in term 1.
I20260619 10:20:54.453032 19056 leader_election.cc:302] T 70935534710b45dd91d2c32320085806 P 720e2ee687d34cdc8425f0d12991b9d2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1d5504d3acf74ce393bad25f554af3c6, 720e2ee687d34cdc8425f0d12991b9d2; no voters:
I20260619 10:20:54.453122 19320 raft_consensus.cc:2781] T 70935534710b45dd91d2c32320085806 P 720e2ee687d34cdc8425f0d12991b9d2 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 10:20:54.453275 19320 raft_consensus.cc:686] T 70935534710b45dd91d2c32320085806 P 720e2ee687d34cdc8425f0d12991b9d2 [term 1 LEADER]: Becoming Leader. State: Replica: 720e2ee687d34cdc8425f0d12991b9d2, State: Running, Role: LEADER
I20260619 10:20:54.453372 19320 consensus_queue.cc:237] T 70935534710b45dd91d2c32320085806 P 720e2ee687d34cdc8425f0d12991b9d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3844982d2d44e3285b8408e1448d94e" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 38291 } } peers { permanent_uuid: "720e2ee687d34cdc8425f0d12991b9d2" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 41331 } } peers { permanent_uuid: "1d5504d3acf74ce393bad25f554af3c6" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 41153 } }
I20260619 10:20:54.454129 18840 catalog_manager.cc:5495] T 70935534710b45dd91d2c32320085806 P 720e2ee687d34cdc8425f0d12991b9d2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 720e2ee687d34cdc8425f0d12991b9d2 (127.0.206.66). New cstate: current_term: 1 leader_uuid: "720e2ee687d34cdc8425f0d12991b9d2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3844982d2d44e3285b8408e1448d94e" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 38291 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "720e2ee687d34cdc8425f0d12991b9d2" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 41331 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1d5504d3acf74ce393bad25f554af3c6" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 41153 } health_report { overall_health: UNKNOWN } } }
W20260619 10:20:54.490267 19301 tablet.cc:2369] T 70935534710b45dd91d2c32320085806 P 1d5504d3acf74ce393bad25f554af3c6: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 10:20:54.500430 19311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:20:54.498078 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39476 (local address 127.0.206.66:41331)
0619 10:20:54.498120 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0619 10:20:54.498122 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 10:20:54.498137 (+ 15us) server_negotiation.cc:407] Connection header received
0619 10:20:54.498192 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:20:54.498193 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:20:54.498210 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:20:54.498262 (+ 52us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 10:20:54.498384 (+ 122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:20:54.498932 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:20:54.499682 (+ 750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:20:54.499880 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:20:54.499932 (+ 52us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:20:54.500140 (+ 208us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 10:20:54.500222 (+ 82us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 10:20:54.500302 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0619 10:20:54.500344 (+ 42us) server_negotiation.cc:299] Negotiation successful
0619 10:20:54.500372 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260619 10:20:54.502202 19254 raft_consensus.cc:1261] T 70935534710b45dd91d2c32320085806 P 1d5504d3acf74ce393bad25f554af3c6 [term 1 FOLLOWER]: Refusing update from remote peer 720e2ee687d34cdc8425f0d12991b9d2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 10:20:54.502303 18986 raft_consensus.cc:1261] T 70935534710b45dd91d2c32320085806 P e3844982d2d44e3285b8408e1448d94e [term 1 FOLLOWER]: Refusing update from remote peer 720e2ee687d34cdc8425f0d12991b9d2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 10:20:54.502537 19320 consensus_queue.cc:1035] T 70935534710b45dd91d2c32320085806 P 720e2ee687d34cdc8425f0d12991b9d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1d5504d3acf74ce393bad25f554af3c6" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 41153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 10:20:54.502718 19324 consensus_queue.cc:1035] T 70935534710b45dd91d2c32320085806 P 720e2ee687d34cdc8425f0d12991b9d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e3844982d2d44e3285b8408e1448d94e" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 38291 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 10:20:54.505045 19327 mvcc.cc:205] Tried to move back new op lower bound from 7298516805638463488 to 7298516805441564672. Current Snapshot: MvccSnapshot[applied={T|T < 7298516805638463488}]
I20260619 10:20:54.505251 19329 mvcc.cc:205] Tried to move back new op lower bound from 7298516805638463488 to 7298516805441564672. Current Snapshot: MvccSnapshot[applied={T|T < 7298516805638463488}]
I20260619 10:20:54.507539 18841 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58446:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 10:20:54.507613 18841 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58446:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 10:20:54.508364 18841 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 52c1c2a8b1ea4577b1b7fb68234b509e: Sending DeleteTablet for 3 replicas of tablet 70935534710b45dd91d2c32320085806
I20260619 10:20:54.508723 19234 tablet_service.cc:1514] Processing DeleteTablet for tablet 70935534710b45dd91d2c32320085806 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 10:20:54 UTC) from {username='kudu', principal='kudu/127.0.206.126@KRBTEST.COM'} at 127.0.0.1:50602
I20260619 10:20:54.508742 18966 tablet_service.cc:1514] Processing DeleteTablet for tablet 70935534710b45dd91d2c32320085806 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 10:20:54 UTC) from {username='kudu', principal='kudu/127.0.206.126@KRBTEST.COM'} at 127.0.0.1:57618
I20260619 10:20:54.508764 19100 tablet_service.cc:1514] Processing DeleteTablet for tablet 70935534710b45dd91d2c32320085806 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 10:20:54 UTC) from {username='kudu', principal='kudu/127.0.206.126@KRBTEST.COM'} at 127.0.0.1:39460
I20260619 10:20:54.509006 19339 tablet_replica.cc:330] T 70935534710b45dd91d2c32320085806 P 720e2ee687d34cdc8425f0d12991b9d2: stopping tablet replica
I20260619 10:20:54.509100 19339 raft_consensus.cc:2229] T 70935534710b45dd91d2c32320085806 P 720e2ee687d34cdc8425f0d12991b9d2 [term 1 LEADER]: Raft consensus shutting down.
I20260619 10:20:54.509181 825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTU9OP6/build/release/bin/kudu with pid 18903
I20260619 10:20:54.509217 19339 raft_consensus.cc:2258] T 70935534710b45dd91d2c32320085806 P 720e2ee687d34cdc8425f0d12991b9d2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 10:20:54.509672 19339 ts_tablet_manager.cc:1905] T 70935534710b45dd91d2c32320085806 P 720e2ee687d34cdc8425f0d12991b9d2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 10:20:54.511929 19339 ts_tablet_manager.cc:1918] T 70935534710b45dd91d2c32320085806 P 720e2ee687d34cdc8425f0d12991b9d2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 10:20:54.512001 19339 log.cc:1196] T 70935534710b45dd91d2c32320085806 P 720e2ee687d34cdc8425f0d12991b9d2: Deleting WAL directory at /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781864224577058-825-0/minicluster-data/ts-1/wal/wals/70935534710b45dd91d2c32320085806
I20260619 10:20:54.512274 19339 ts_tablet_manager.cc:1939] T 70935534710b45dd91d2c32320085806 P 720e2ee687d34cdc8425f0d12991b9d2: Deleting consensus metadata
I20260619 10:20:54.512641 18826 catalog_manager.cc:4841] TS 720e2ee687d34cdc8425f0d12991b9d2 (127.0.206.66:41331): tablet 70935534710b45dd91d2c32320085806 (table test-table [id=2264b0edb04f4e7d9aedd905115f7d47]) successfully deleted
I20260619 10:20:54.514945 19337 tablet_replica.cc:330] T 70935534710b45dd91d2c32320085806 P 1d5504d3acf74ce393bad25f554af3c6: stopping tablet replica
I20260619 10:20:54.515048 19337 raft_consensus.cc:2229] T 70935534710b45dd91d2c32320085806 P 1d5504d3acf74ce393bad25f554af3c6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 10:20:54.515143 19337 raft_consensus.cc:2258] T 70935534710b45dd91d2c32320085806 P 1d5504d3acf74ce393bad25f554af3c6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 10:20:54.515578 19337 ts_tablet_manager.cc:1905] T 70935534710b45dd91d2c32320085806 P 1d5504d3acf74ce393bad25f554af3c6: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260619 10:20:54.516446 18828 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.0.206.65:38291 (error 108)
W20260619 10:20:54.516880 19309 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 10:20:54.516740 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.65:38291 (local address 127.0.0.1:57622)
0619 10:20:54.516806 (+ 66us) negotiation.cc:107] Waiting for socket to connect
0619 10:20:54.516838 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.65:38291: connect: Connection reset by peer (error 104)
Metrics: {"client-negotiator.queue_time_us":42}
I20260619 10:20:54.517074 19337 ts_tablet_manager.cc:1918] T 70935534710b45dd91d2c32320085806 P 1d5504d3acf74ce393bad25f554af3c6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
W20260619 10:20:54.517077 18828 catalog_manager.cc:4584] TS e3844982d2d44e3285b8408e1448d94e (127.0.206.65:38291): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 70935534710b45dd91d2c32320085806: Network error: Client connection negotiation failed: client connection to 127.0.206.65:38291: connect: Connection reset by peer (error 104)
I20260619 10:20:54.517143 19337 log.cc:1196] T 70935534710b45dd91d2c32320085806 P 1d5504d3acf74ce393bad25f554af3c6: Deleting WAL directory at /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781864224577058-825-0/minicluster-data/ts-2/wal/wals/70935534710b45dd91d2c32320085806
I20260619 10:20:54.517258 825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTU9OP6/build/release/bin/kudu with pid 19037
I20260619 10:20:54.517500 19337 ts_tablet_manager.cc:1939] T 70935534710b45dd91d2c32320085806 P 1d5504d3acf74ce393bad25f554af3c6: Deleting consensus metadata
I20260619 10:20:54.519093 18828 catalog_manager.cc:4841] TS 1d5504d3acf74ce393bad25f554af3c6 (127.0.206.67:41153): tablet 70935534710b45dd91d2c32320085806 (table test-table [id=2264b0edb04f4e7d9aedd905115f7d47]) successfully deleted
I20260619 10:20:54.521718 825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTU9OP6/build/release/bin/kudu with pid 19171
I20260619 10:20:54.525887 825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTU9OP6/build/release/bin/kudu with pid 18809
2026-06-19T10:20:54Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (2027 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-06-19T10:20:54Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T10:20:54Z Disabled control of system clock
I20260619 10:20:54.551834 825 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTU9OP6/build/release/bin/kudu
/tmp/dist-test-taskTU9OP6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781864224577058-825-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781864224577058-825-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781864224577058-825-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781864224577058-825-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.206.126:38693
--webserver_interface=127.0.206.126
--webserver_port=0
--builtin_ntp_servers=127.0.206.84:39951
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.206.126:38693
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260619 10:20:54.629832 19344 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 10:20:54.630036 19344 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 10:20:54.630069 19344 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 10:20:54.631148 19344 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 10:20:54.631186 19344 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 10:20:54.631197 19344 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 10:20:54.631209 19344 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 10:20:54.631222 19344 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 10:20:54.632408 19344 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.206.84:39951
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781864224577058-825-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781864224577058-825-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.206.126:38693
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.206.126:38693
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781864224577058-825-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.206.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19344
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781864224577058-825-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 09:32:44 UTC on ccycloud.quasar-gbtldp.root.comops.site
build id 4455440
I20260619 10:20:54.632596 19344 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 10:20:54.632807 19344 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260619 10:20:54.637535 19344 server_base.cc:1034] running on GCE node
W20260619 10:20:54.637456 19350 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 10:20:54.637427 19349 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 10:20:54.637410 19352 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 10:20:54.638007 19344 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 10:20:54.638299 19344 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 10:20:54.639467 19344 hybrid_clock.cc:648] HybridClock initialized: now 1781864454639443 us; error 36 us; skew 500 ppm
I20260619 10:20:54.640597 19344 webserver.cc:460] Webserver started at http://127.0.206.126:34993/ using document root <none> and password file <none>
I20260619 10:20:54.640851 19344 fs_manager.cc:359] Metadata directory not provided
I20260619 10:20:54.640923 19344 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 10:20:54.641064 19344 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 10:20:54.642076 19344 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781864224577058-825-0/minicluster-data/master-0/data/instance:
uuid: "3cddd035221d435f91f59c4d11a1647f"
format_stamp: "Formatted at 2026-06-19 10:20:54 on dist-test-slave-6pt2"
server_key: "833816ca1ce9cd8a185122ea9e358f35"
server_key_iv: "d9c5ef750d2a526ae3bc29f496b1f3ea"
server_key_version: "encryptionkey@0"
I20260619 10:20:54.642409 19344 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781864224577058-825-0/minicluster-data/master-0/wal/instance:
uuid: "3cddd035221d435f91f59c4d11a1647f"
format_stamp: "Formatted at 2026-06-19 10:20:54 on dist-test-slave-6pt2"
server_key: "833816ca1ce9cd8a185122ea9e358f35"
server_key_iv: "d9c5ef750d2a526ae3bc29f496b1f3ea"
server_key_version: "encryptionkey@0"
I20260619 10:20:54.643677 19344 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 10:20:54.644344 19358 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 10:20:54.644521 19344 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 10:20:54.644613 19344 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781864224577058-825-0/minicluster-data/master-0/data,/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781864224577058-825-0/minicluster-data/master-0/wal
uuid: "3cddd035221d435f91f59c4d11a1647f"
format_stamp: "Formatted at 2026-06-19 10:20:54 on dist-test-slave-6pt2"
server_key: "833816ca1ce9cd8a185122ea9e358f35"
server_key_iv: "d9c5ef750d2a526ae3bc29f496b1f3ea"
server_key_version: "encryptionkey@0"
I20260619 10:20:54.644687 19344 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781864224577058-825-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781864224577058-825-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781864224577058-825-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 10:20:54.660343 19344 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 10:20:54.660642 19344 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 10:20:54.660801 19344 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 10:20:54.664655 19344 rpc_server.cc:307] RPC server started. Bound to: 127.0.206.126:38693
I20260619 10:20:54.664685 19410 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.206.126:38693 every 8 connection(s)
I20260619 10:20:54.665010 19344 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781864224577058-825-0/minicluster-data/master-0/data/info.pb
I20260619 10:20:54.665337 825 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTU9OP6/build/release/bin/kudu as pid 19344
I20260619 10:20:54.665442 825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781864224577058-825-0/minicluster-data/master-0/wal/instance
I20260619 10:20:54.665642 825 external_mini_cluster.cc:1428] Setting key a9123ce036c3e7a0327b08c0b41fa51f
I20260619 10:20:54.666105 19411 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 10:20:54.668748 19411 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3cddd035221d435f91f59c4d11a1647f: Bootstrap starting.
I20260619 10:20:54.669304 19411 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3cddd035221d435f91f59c4d11a1647f: Neither blocks nor log segments found. Creating new log.
I20260619 10:20:54.669584 19411 log.cc:824] T 00000000000000000000000000000000 P 3cddd035221d435f91f59c4d11a1647f: Log is configured to *not* fsync() on all Append() calls
I20260619 10:20:54.670312 19411 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3cddd035221d435f91f59c4d11a1647f: No bootstrap required, opened a new log
I20260619 10:20:54.670974 19414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:20:54.666159 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54104 (local address 127.0.206.126:38693)
0619 10:20:54.666367 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0619 10:20:54.666382 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0619 10:20:54.666405 (+ 23us) server_negotiation.cc:407] Connection header received
0619 10:20:54.666898 (+ 493us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:20:54.666905 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:20:54.666966 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:20:54.667101 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:20:54.667319 (+ 218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:20:54.668195 (+ 876us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:20:54.669018 (+ 823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:20:54.669453 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:20:54.669582 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:20:54.670290 (+ 708us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 10:20:54.670294 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 10:20:54.670295 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 10:20:54.670500 (+ 205us) server_negotiation.cc:922] calling sasl_server_start()
0619 10:20:54.670544 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 10:20:54.670555 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 10:20:54.670654 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0619 10:20:54.670729 (+ 75us) server_negotiation.cc:299] Negotiation successful
0619 10:20:54.670804 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":132,"thread_start_us":84,"threads_started":1}
I20260619 10:20:54.672397 19411 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3cddd035221d435f91f59c4d11a1647f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3cddd035221d435f91f59c4d11a1647f" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 38693 } }
I20260619 10:20:54.672546 19411 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3cddd035221d435f91f59c4d11a1647f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 10:20:54.672591 19411 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3cddd035221d435f91f59c4d11a1647f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3cddd035221d435f91f59c4d11a1647f, State: Initialized, Role: FOLLOWER
I20260619 10:20:54.672708 19411 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3cddd035221d435f91f59c4d11a1647f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3cddd035221d435f91f59c4d11a1647f" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 38693 } }
I20260619 10:20:54.672785 19411 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3cddd035221d435f91f59c4d11a1647f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 10:20:54.672825 19411 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3cddd035221d435f91f59c4d11a1647f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 10:20:54.672873 19411 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3cddd035221d435f91f59c4d11a1647f [term 0 FOLLOWER]: Advancing to term 1
I20260619 10:20:54.673374 19411 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3cddd035221d435f91f59c4d11a1647f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3cddd035221d435f91f59c4d11a1647f" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 38693 } }
I20260619 10:20:54.673484 19411 leader_election.cc:302] T 00000000000000000000000000000000 P 3cddd035221d435f91f59c4d11a1647f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3cddd035221d435f91f59c4d11a1647f; no voters:
I20260619 10:20:54.673661 19411 leader_election.cc:288] T 00000000000000000000000000000000 P 3cddd035221d435f91f59c4d11a1647f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 10:20:54.673712 19416 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3cddd035221d435f91f59c4d11a1647f [term 1 FOLLOWER]: Leader election won for term 1
I20260619 10:20:54.673874 19416 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3cddd035221d435f91f59c4d11a1647f [term 1 LEADER]: Becoming Leader. State: Replica: 3cddd035221d435f91f59c4d11a1647f, State: Running, Role: LEADER
I20260619 10:20:54.673910 19411 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3cddd035221d435f91f59c4d11a1647f [sys.catalog]: configured and running, proceeding with master startup.
I20260619 10:20:54.674026 19416 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3cddd035221d435f91f59c4d11a1647f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3cddd035221d435f91f59c4d11a1647f" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 38693 } }
I20260619 10:20:54.674348 19417 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3cddd035221d435f91f59c4d11a1647f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3cddd035221d435f91f59c4d11a1647f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3cddd035221d435f91f59c4d11a1647f" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 38693 } } }
I20260619 10:20:54.674357 19418 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3cddd035221d435f91f59c4d11a1647f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3cddd035221d435f91f59c4d11a1647f. Latest consensus state: current_term: 1 leader_uuid: "3cddd035221d435f91f59c4d11a1647f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3cddd035221d435f91f59c4d11a1647f" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 38693 } } }
I20260619 10:20:54.674547 19418 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3cddd035221d435f91f59c4d11a1647f [sys.catalog]: This master's current role is: LEADER
I20260619 10:20:54.674485 19417 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3cddd035221d435f91f59c4d11a1647f [sys.catalog]: This master's current role is: LEADER
I20260619 10:20:54.674813 19425 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 10:20:54.675254 19425 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 10:20:54.676584 19425 catalog_manager.cc:1347] Generated new cluster ID: 60e71c43829145f895b4a28a322c2b1c
I20260619 10:20:54.676635 19425 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 10:20:54.682578 19425 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 10:20:54.683056 19425 catalog_manager.cc:1504] Loading token signing keys...
I20260619 10:20:54.691098 19425 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3cddd035221d435f91f59c4d11a1647f: Generated new TSK 0
I20260619 10:20:54.691250 19425 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 10:20:54.697883 825 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTU9OP6/build/release/bin/kudu
/tmp/dist-test-taskTU9OP6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781864224577058-825-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781864224577058-825-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781864224577058-825-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781864224577058-825-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.206.65:0
--local_ip_for_outbound_sockets=127.0.206.65
--webserver_interface=127.0.206.65
--webserver_port=0
--tserver_master_addrs=127.0.206.126:38693
--builtin_ntp_servers=127.0.206.84:39951
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260619 10:20:54.797179 19435 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 10:20:54.797386 19435 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 10:20:54.797407 19435 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 10:20:54.798516 19435 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 10:20:54.798552 19435 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 10:20:54.798575 19435 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.206.65
I20260619 10:20:54.799760 19435 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.206.84:39951
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781864224577058-825-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781864224577058-825-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.206.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781864224577058-825-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.206.65
--webserver_port=0
--tserver_master_addrs=127.0.206.126:38693
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19435
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.206.65
--log_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781864224577058-825-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 09:32:44 UTC on ccycloud.quasar-gbtldp.root.comops.site
build id 4455440
I20260619 10:20:54.799962 19435 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 10:20:54.800223 19435 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 10:20:54.805795 19443 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 10:20:54.805805 19441 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 10:20:54.806031 19435 server_base.cc:1034] running on GCE node
W20260619 10:20:54.805850 19440 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 10:20:54.806241 19435 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 10:20:54.806483 19435 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 10:20:54.807650 19435 hybrid_clock.cc:648] HybridClock initialized: now 1781864454807638 us; error 31 us; skew 500 ppm
I20260619 10:20:54.808781 19435 webserver.cc:460] Webserver started at http://127.0.206.65:38365/ using document root <none> and password file <none>
I20260619 10:20:54.809002 19435 fs_manager.cc:359] Metadata directory not provided
I20260619 10:20:54.809085 19435 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 10:20:54.809213 19435 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 10:20:54.810186 19435 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781864224577058-825-0/minicluster-data/ts-0/data/instance:
uuid: "73b572f1a86d416691e3a7ef1cbc8f47"
format_stamp: "Formatted at 2026-06-19 10:20:54 on dist-test-slave-6pt2"
server_key: "d275131184c009a137ab89338c419d76"
server_key_iv: "82a24863c7d5517a5dc5fea49f0f435c"
server_key_version: "encryptionkey@0"
I20260619 10:20:54.810519 19435 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781864224577058-825-0/minicluster-data/ts-0/wal/instance:
uuid: "73b572f1a86d416691e3a7ef1cbc8f47"
format_stamp: "Formatted at 2026-06-19 10:20:54 on dist-test-slave-6pt2"
server_key: "d275131184c009a137ab89338c419d76"
server_key_iv: "82a24863c7d5517a5dc5fea49f0f435c"
server_key_version: "encryptionkey@0"
I20260619 10:20:54.811754 19435 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260619 10:20:54.812541 19449 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 10:20:54.812762 19435 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260619 10:20:54.812861 19435 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781864224577058-825-0/minicluster-data/ts-0/data,/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781864224577058-825-0/minicluster-data/ts-0/wal
uuid: "73b572f1a86d416691e3a7ef1cbc8f47"
format_stamp: "Formatted at 2026-06-19 10:20:54 on dist-test-slave-6pt2"
server_key: "d275131184c009a137ab89338c419d76"
server_key_iv: "82a24863c7d5517a5dc5fea49f0f435c"
server_key_version: "encryptionkey@0"
I20260619 10:20:54.812945 19435 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781864224577058-825-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781864224577058-825-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781864224577058-825-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 10:20:54.842129 19435 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 10:20:54.842449 19435 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 10:20:54.842643 19435 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 10:20:54.842885 19435 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 10:20:54.843204 19435 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 10:20:54.843254 19435 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 10:20:54.843320 19435 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 10:20:54.843348 19435 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 10:20:54.848889 19435 rpc_server.cc:307] RPC server started. Bound to: 127.0.206.65:43293
I20260619 10:20:54.848985 19562 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.206.65:43293 every 8 connection(s)
I20260619 10:20:54.849261 19435 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781864224577058-825-0/minicluster-data/ts-0/data/info.pb
I20260619 10:20:54.853246 825 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTU9OP6/build/release/bin/kudu as pid 19435
I20260619 10:20:54.853367 825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781864224577058-825-0/minicluster-data/ts-0/wal/instance
I20260619 10:20:54.853557 825 external_mini_cluster.cc:1428] Setting key f85f393baeea238b1d81a319a66bb75c
I20260619 10:20:54.854475 19414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:20:54.849925 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.65:55921 (local address 127.0.206.126:38693)
0619 10:20:54.849987 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0619 10:20:54.849989 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 10:20:54.850256 (+ 267us) server_negotiation.cc:407] Connection header received
0619 10:20:54.850871 (+ 615us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:20:54.850873 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:20:54.850888 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:20:54.850951 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:20:54.851574 (+ 623us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:20:54.852046 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:20:54.852915 (+ 869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:20:54.853123 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:20:54.853199 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:20:54.854141 (+ 942us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 10:20:54.854143 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 10:20:54.854143 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 10:20:54.854147 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0619 10:20:54.854164 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 10:20:54.854169 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 10:20:54.854222 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0619 10:20:54.854318 (+ 96us) server_negotiation.cc:299] Negotiation successful
0619 10:20:54.854354 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260619 10:20:54.854576 825 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTU9OP6/build/release/bin/kudu
/tmp/dist-test-taskTU9OP6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781864224577058-825-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781864224577058-825-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781864224577058-825-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781864224577058-825-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.206.66:0
--local_ip_for_outbound_sockets=127.0.206.66
--webserver_interface=127.0.206.66
--webserver_port=0
--tserver_master_addrs=127.0.206.126:38693
--builtin_ntp_servers=127.0.206.84:39951
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260619 10:20:54.854624 19565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:20:54.849933 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:38693 (local address 127.0.206.65:55921)
0619 10:20:54.850144 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0619 10:20:54.850174 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0619 10:20:54.850766 (+ 592us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 10:20:54.851016 (+ 250us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 10:20:54.851020 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 10:20:54.851034 (+ 14us) client_negotiation.cc:189] Negotiated authn=SASL
0619 10:20:54.851490 (+ 456us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:20:54.851493 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:20:54.852149 (+ 656us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:20:54.852150 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:20:54.852836 (+ 686us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:20:54.852838 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:20:54.853209 (+ 371us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:20:54.853210 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:20:54.853413 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:20:54.853992 (+ 579us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 10:20:54.853993 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0619 10:20:54.854027 (+ 34us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 10:20:54.854051 (+ 24us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 10:20:54.854235 (+ 184us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 10:20:54.854241 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 10:20:54.854244 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 10:20:54.854320 (+ 76us) client_negotiation.cc:240] Negotiation successful
0619 10:20:54.854411 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":157,"thread_start_us":110,"threads_started":1}
I20260619 10:20:54.855052 19563 heartbeater.cc:347] Connected to a master server at 127.0.206.126:38693
I20260619 10:20:54.855150 19563 heartbeater.cc:464] Registering TS with master...
I20260619 10:20:54.855310 19563 heartbeater.cc:511] Master 127.0.206.126:38693 requested a full tablet report, sending...
I20260619 10:20:54.855753 19375 ts_manager.cc:194] Registered new tserver with Master: 73b572f1a86d416691e3a7ef1cbc8f47 (127.0.206.65:43293)
I20260619 10:20:54.856315 19375 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.206.65:55921
W20260619 10:20:54.931939 19566 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 10:20:54.932116 19566 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 10:20:54.932137 19566 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 10:20:54.933252 19566 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 10:20:54.933306 19566 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 10:20:54.933331 19566 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.206.66
I20260619 10:20:54.934682 19566 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.206.84:39951
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781864224577058-825-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781864224577058-825-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.206.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781864224577058-825-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.206.66
--webserver_port=0
--tserver_master_addrs=127.0.206.126:38693
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19566
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.206.66
--log_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781864224577058-825-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 09:32:44 UTC on ccycloud.quasar-gbtldp.root.comops.site
build id 4455440
I20260619 10:20:54.934890 19566 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 10:20:54.935158 19566 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 10:20:54.939833 19574 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 10:20:54.939882 19572 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 10:20:54.939831 19571 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 10:20:54.939924 19566 server_base.cc:1034] running on GCE node
I20260619 10:20:54.940201 19566 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 10:20:54.940500 19566 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 10:20:54.941660 19566 hybrid_clock.cc:648] HybridClock initialized: now 1781864454941645 us; error 42 us; skew 500 ppm
I20260619 10:20:54.942925 19566 webserver.cc:460] Webserver started at http://127.0.206.66:38751/ using document root <none> and password file <none>
I20260619 10:20:54.943149 19566 fs_manager.cc:359] Metadata directory not provided
I20260619 10:20:54.943215 19566 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 10:20:54.943353 19566 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 10:20:54.944363 19566 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781864224577058-825-0/minicluster-data/ts-1/data/instance:
uuid: "552783c35ce14f7dbb694cfe5fa15d9c"
format_stamp: "Formatted at 2026-06-19 10:20:54 on dist-test-slave-6pt2"
server_key: "802af703301e318d28ca8a1c0343274b"
server_key_iv: "d811b15eda3e0421a934543d056384da"
server_key_version: "encryptionkey@0"
I20260619 10:20:54.944708 19566 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781864224577058-825-0/minicluster-data/ts-1/wal/instance:
uuid: "552783c35ce14f7dbb694cfe5fa15d9c"
format_stamp: "Formatted at 2026-06-19 10:20:54 on dist-test-slave-6pt2"
server_key: "802af703301e318d28ca8a1c0343274b"
server_key_iv: "d811b15eda3e0421a934543d056384da"
server_key_version: "encryptionkey@0"
I20260619 10:20:54.945972 19566 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260619 10:20:54.946693 19580 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 10:20:54.946877 19566 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260619 10:20:54.946986 19566 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781864224577058-825-0/minicluster-data/ts-1/data,/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781864224577058-825-0/minicluster-data/ts-1/wal
uuid: "552783c35ce14f7dbb694cfe5fa15d9c"
format_stamp: "Formatted at 2026-06-19 10:20:54 on dist-test-slave-6pt2"
server_key: "802af703301e318d28ca8a1c0343274b"
server_key_iv: "d811b15eda3e0421a934543d056384da"
server_key_version: "encryptionkey@0"
I20260619 10:20:54.947069 19566 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781864224577058-825-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781864224577058-825-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781864224577058-825-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 10:20:54.970834 19566 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 10:20:54.971102 19566 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 10:20:54.971244 19566 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 10:20:54.971457 19566 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 10:20:54.971781 19566 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 10:20:54.971813 19566 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 10:20:54.971863 19566 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 10:20:54.971889 19566 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 10:20:54.977440 19566 rpc_server.cc:307] RPC server started. Bound to: 127.0.206.66:40061
I20260619 10:20:54.977517 19693 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.206.66:40061 every 8 connection(s)
I20260619 10:20:54.977820 19566 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781864224577058-825-0/minicluster-data/ts-1/data/info.pb
I20260619 10:20:54.979161 825 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTU9OP6/build/release/bin/kudu as pid 19566
I20260619 10:20:54.979286 825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781864224577058-825-0/minicluster-data/ts-1/wal/instance
I20260619 10:20:54.979534 825 external_mini_cluster.cc:1428] Setting key aa00dd291a341ba702e0a03629690d61
I20260619 10:20:54.980479 825 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTU9OP6/build/release/bin/kudu
/tmp/dist-test-taskTU9OP6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781864224577058-825-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781864224577058-825-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781864224577058-825-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781864224577058-825-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.206.67:0
--local_ip_for_outbound_sockets=127.0.206.67
--webserver_interface=127.0.206.67
--webserver_port=0
--tserver_master_addrs=127.0.206.126:38693
--builtin_ntp_servers=127.0.206.84:39951
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260619 10:20:54.983100 19414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:20:54.978542 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.66:39009 (local address 127.0.206.126:38693)
0619 10:20:54.978648 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0619 10:20:54.978650 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 10:20:54.978835 (+ 185us) server_negotiation.cc:407] Connection header received
0619 10:20:54.979371 (+ 536us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:20:54.979373 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:20:54.979386 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:20:54.979454 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:20:54.980121 (+ 667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:20:54.980847 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:20:54.981754 (+ 907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:20:54.981962 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:20:54.982038 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:20:54.982849 (+ 811us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 10:20:54.982850 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 10:20:54.982851 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 10:20:54.982855 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0619 10:20:54.982867 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 10:20:54.982871 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 10:20:54.982912 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0619 10:20:54.983006 (+ 94us) server_negotiation.cc:299] Negotiation successful
0619 10:20:54.983028 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260619 10:20:54.983335 19696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:20:54.978525 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:38693 (local address 127.0.206.66:39009)
0619 10:20:54.978701 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0619 10:20:54.978744 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0619 10:20:54.979243 (+ 499us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 10:20:54.979513 (+ 270us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 10:20:54.979517 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 10:20:54.979524 (+ 7us) client_negotiation.cc:189] Negotiated authn=SASL
0619 10:20:54.980030 (+ 506us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:20:54.980035 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:20:54.980941 (+ 906us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:20:54.980941 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:20:54.981664 (+ 723us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:20:54.981666 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:20:54.982046 (+ 380us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:20:54.982048 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:20:54.982177 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:20:54.982712 (+ 535us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 10:20:54.982713 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0619 10:20:54.982739 (+ 26us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 10:20:54.982758 (+ 19us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 10:20:54.982917 (+ 159us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 10:20:54.982921 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 10:20:54.982923 (+ 2us) client_negotiation.cc:769] Sending connection context
0619 10:20:54.982996 (+ 73us) client_negotiation.cc:240] Negotiation successful
0619 10:20:54.983085 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":89,"threads_started":1}
I20260619 10:20:54.983754 19694 heartbeater.cc:347] Connected to a master server at 127.0.206.126:38693
I20260619 10:20:54.983858 19694 heartbeater.cc:464] Registering TS with master...
I20260619 10:20:54.984055 19694 heartbeater.cc:511] Master 127.0.206.126:38693 requested a full tablet report, sending...
I20260619 10:20:54.984521 19375 ts_manager.cc:194] Registered new tserver with Master: 552783c35ce14f7dbb694cfe5fa15d9c (127.0.206.66:40061)
I20260619 10:20:54.985095 19375 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.206.66:39009
W20260619 10:20:55.059104 19697 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 10:20:55.059279 19697 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 10:20:55.059311 19697 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 10:20:55.060341 19697 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 10:20:55.060372 19697 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 10:20:55.060395 19697 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.206.67
I20260619 10:20:55.061607 19697 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.206.84:39951
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781864224577058-825-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781864224577058-825-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.206.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781864224577058-825-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.206.67
--webserver_port=0
--tserver_master_addrs=127.0.206.126:38693
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19697
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.206.67
--log_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781864224577058-825-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 09:32:44 UTC on ccycloud.quasar-gbtldp.root.comops.site
build id 4455440
I20260619 10:20:55.061785 19697 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 10:20:55.062033 19697 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260619 10:20:55.066738 19697 server_base.cc:1034] running on GCE node
W20260619 10:20:55.066702 19702 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 10:20:55.066712 19703 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 10:20:55.066702 19705 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 10:20:55.067035 19697 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 10:20:55.067241 19697 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 10:20:55.068389 19697 hybrid_clock.cc:648] HybridClock initialized: now 1781864455068367 us; error 54 us; skew 500 ppm
I20260619 10:20:55.069555 19697 webserver.cc:460] Webserver started at http://127.0.206.67:34095/ using document root <none> and password file <none>
I20260619 10:20:55.069784 19697 fs_manager.cc:359] Metadata directory not provided
I20260619 10:20:55.069828 19697 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 10:20:55.069985 19697 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 10:20:55.070981 19697 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781864224577058-825-0/minicluster-data/ts-2/data/instance:
uuid: "6d841a2baa274687a27761a5c8eb77bc"
format_stamp: "Formatted at 2026-06-19 10:20:55 on dist-test-slave-6pt2"
server_key: "d5293e5007476780353191e5b69bdbcc"
server_key_iv: "dc7addabf463d342326484ce167be707"
server_key_version: "encryptionkey@0"
I20260619 10:20:55.071296 19697 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781864224577058-825-0/minicluster-data/ts-2/wal/instance:
uuid: "6d841a2baa274687a27761a5c8eb77bc"
format_stamp: "Formatted at 2026-06-19 10:20:55 on dist-test-slave-6pt2"
server_key: "d5293e5007476780353191e5b69bdbcc"
server_key_iv: "dc7addabf463d342326484ce167be707"
server_key_version: "encryptionkey@0"
I20260619 10:20:55.072520 19697 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260619 10:20:55.073172 19711 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 10:20:55.073349 19697 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260619 10:20:55.073432 19697 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781864224577058-825-0/minicluster-data/ts-2/data,/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781864224577058-825-0/minicluster-data/ts-2/wal
uuid: "6d841a2baa274687a27761a5c8eb77bc"
format_stamp: "Formatted at 2026-06-19 10:20:55 on dist-test-slave-6pt2"
server_key: "d5293e5007476780353191e5b69bdbcc"
server_key_iv: "dc7addabf463d342326484ce167be707"
server_key_version: "encryptionkey@0"
I20260619 10:20:55.073500 19697 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781864224577058-825-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781864224577058-825-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781864224577058-825-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 10:20:55.087445 19697 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 10:20:55.087716 19697 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 10:20:55.087853 19697 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 10:20:55.088074 19697 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 10:20:55.088379 19697 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 10:20:55.088434 19697 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 10:20:55.088474 19697 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 10:20:55.088500 19697 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 10:20:55.094116 19697 rpc_server.cc:307] RPC server started. Bound to: 127.0.206.67:37791
I20260619 10:20:55.094197 19824 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.206.67:37791 every 8 connection(s)
I20260619 10:20:55.094503 19697 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781864224577058-825-0/minicluster-data/ts-2/data/info.pb
I20260619 10:20:55.094925 825 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTU9OP6/build/release/bin/kudu as pid 19697
I20260619 10:20:55.095028 825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781864224577058-825-0/minicluster-data/ts-2/wal/instance
I20260619 10:20:55.095261 825 external_mini_cluster.cc:1428] Setting key ff03147a2d6d4daa1f1bbbcf9cb1f1e6
I20260619 10:20:55.099675 19414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:20:55.095297 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.67:34345 (local address 127.0.206.126:38693)
0619 10:20:55.095354 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0619 10:20:55.095357 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 10:20:55.095574 (+ 217us) server_negotiation.cc:407] Connection header received
0619 10:20:55.096267 (+ 693us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:20:55.096268 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:20:55.096279 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:20:55.096310 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:20:55.097018 (+ 708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:20:55.097514 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:20:55.098420 (+ 906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:20:55.098590 (+ 170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:20:55.098696 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:20:55.099402 (+ 706us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 10:20:55.099403 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 10:20:55.099404 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 10:20:55.099407 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0619 10:20:55.099420 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 10:20:55.099423 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 10:20:55.099465 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0619 10:20:55.099560 (+ 95us) server_negotiation.cc:299] Negotiation successful
0619 10:20:55.099581 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260619 10:20:55.099865 19827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:20:55.095277 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:38693 (local address 127.0.206.67:34345)
0619 10:20:55.095458 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0619 10:20:55.095498 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0619 10:20:55.096132 (+ 634us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 10:20:55.096470 (+ 338us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 10:20:55.096473 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 10:20:55.096480 (+ 7us) client_negotiation.cc:189] Negotiated authn=SASL
0619 10:20:55.096943 (+ 463us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:20:55.096947 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:20:55.097596 (+ 649us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:20:55.097597 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:20:55.098350 (+ 753us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:20:55.098352 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:20:55.098701 (+ 349us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:20:55.098702 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:20:55.098814 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:20:55.099269 (+ 455us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 10:20:55.099269 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0619 10:20:55.099294 (+ 25us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 10:20:55.099326 (+ 32us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 10:20:55.099491 (+ 165us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 10:20:55.099495 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 10:20:55.099497 (+ 2us) client_negotiation.cc:769] Sending connection context
0619 10:20:55.099545 (+ 48us) client_negotiation.cc:240] Negotiation successful
0619 10:20:55.099634 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":83,"threads_started":1}
I20260619 10:20:55.100414 19825 heartbeater.cc:347] Connected to a master server at 127.0.206.126:38693
I20260619 10:20:55.100519 19825 heartbeater.cc:464] Registering TS with master...
I20260619 10:20:55.100684 19825 heartbeater.cc:511] Master 127.0.206.126:38693 requested a full tablet report, sending...
I20260619 10:20:55.101030 19375 ts_manager.cc:194] Registered new tserver with Master: 6d841a2baa274687a27761a5c8eb77bc (127.0.206.67:37791)
I20260619 10:20:55.101359 19375 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.206.67:34345
I20260619 10:20:55.107128 825 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 10:20:55.111434 19414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:20:55.108411 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54116 (local address 127.0.206.126:38693)
0619 10:20:55.108469 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0619 10:20:55.108471 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 10:20:55.108574 (+ 103us) server_negotiation.cc:407] Connection header received
0619 10:20:55.108636 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:20:55.108637 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:20:55.108647 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:20:55.108689 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:20:55.108876 (+ 187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:20:55.109363 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:20:55.110118 (+ 755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:20:55.110324 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:20:55.110398 (+ 74us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:20:55.111137 (+ 739us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 10:20:55.111139 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 10:20:55.111139 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 10:20:55.111143 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0619 10:20:55.111158 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 10:20:55.111161 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 10:20:55.111278 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0619 10:20:55.111352 (+ 74us) server_negotiation.cc:299] Negotiation successful
0619 10:20:55.111362 (+ 10us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260619 10:20:55.113005 19375 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:54116:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 10:20:55.114424 19375 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 10:20:55.120381 19835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:20:55.116556 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.67:37791 (local address 127.0.0.1:52136)
0619 10:20:55.116814 (+ 258us) negotiation.cc:107] Waiting for socket to connect
0619 10:20:55.116840 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0619 10:20:55.116880 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 10:20:55.117180 (+ 300us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 10:20:55.117180 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 10:20:55.117183 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 10:20:55.117241 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:20:55.117243 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:20:55.118672 (+ 1429us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:20:55.118673 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:20:55.119479 (+ 806us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:20:55.119482 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:20:55.120102 (+ 620us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:20:55.120103 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:20:55.120172 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:20:55.120183 (+ 11us) client_negotiation.cc:769] Sending connection context
0619 10:20:55.120249 (+ 66us) client_negotiation.cc:240] Negotiation successful
0619 10:20:55.120296 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":242,"mutex_wait_us":6,"thread_start_us":64,"threads_started":1}
I20260619 10:20:55.120393 19837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:20:55.116796 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52136 (local address 127.0.206.67:37791)
0619 10:20:55.117013 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0619 10:20:55.117017 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 10:20:55.117025 (+ 8us) server_negotiation.cc:407] Connection header received
0619 10:20:55.117059 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:20:55.117061 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:20:55.117074 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:20:55.117175 (+ 101us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 10:20:55.117497 (+ 322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:20:55.118553 (+ 1056us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:20:55.119562 (+ 1009us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:20:55.120016 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:20:55.120102 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:20:55.120183 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0619 10:20:55.120259 (+ 76us) server_negotiation.cc:299] Negotiation successful
0619 10:20:55.120295 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":185,"thread_start_us":95,"threads_started":1}
I20260619 10:20:55.120714 19833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:20:55.116556 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.65:43293 (local address 127.0.0.1:36136)
0619 10:20:55.116673 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0619 10:20:55.116712 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0619 10:20:55.116774 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 10:20:55.117142 (+ 368us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 10:20:55.117145 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 10:20:55.117148 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 10:20:55.117223 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:20:55.117225 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:20:55.118627 (+ 1402us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:20:55.118628 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:20:55.119645 (+ 1017us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:20:55.119647 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:20:55.120521 (+ 874us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:20:55.120522 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:20:55.120572 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:20:55.120575 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 10:20:55.120604 (+ 29us) client_negotiation.cc:240] Negotiation successful
0619 10:20:55.120639 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":91,"mutex_wait_us":39,"spinlock_wait_cycles":768,"thread_start_us":93,"threads_started":1}
I20260619 10:20:55.120922 19836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:20:55.116771 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36136 (local address 127.0.206.65:43293)
0619 10:20:55.116956 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0619 10:20:55.116961 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 10:20:55.116972 (+ 11us) server_negotiation.cc:407] Connection header received
0619 10:20:55.117012 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:20:55.117015 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:20:55.117032 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:20:55.117337 (+ 305us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 10:20:55.117407 (+ 70us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:20:55.118517 (+ 1110us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:20:55.119724 (+ 1207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:20:55.120425 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:20:55.120679 (+ 254us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:20:55.120763 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0619 10:20:55.120791 (+ 28us) server_negotiation.cc:299] Negotiation successful
0619 10:20:55.120830 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":142,"thread_start_us":105,"threads_started":1}
I20260619 10:20:55.121273 19759 tablet_service.cc:1467] Processing CreateTablet for tablet 1fa4f0ebcc36456f9ee4fd729f7d21f7 (DEFAULT_TABLE table=test-table [id=e3d98baab1514a008af419cf84d532d6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 10:20:55.121470 19497 tablet_service.cc:1467] Processing CreateTablet for tablet 1fa4f0ebcc36456f9ee4fd729f7d21f7 (DEFAULT_TABLE table=test-table [id=e3d98baab1514a008af419cf84d532d6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 10:20:55.121589 19759 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1fa4f0ebcc36456f9ee4fd729f7d21f7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 10:20:55.121850 19497 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1fa4f0ebcc36456f9ee4fd729f7d21f7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 10:20:55.122750 19834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:20:55.116574 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.66:40061 (local address 127.0.0.1:33804)
0619 10:20:55.117000 (+ 426us) negotiation.cc:107] Waiting for socket to connect
0619 10:20:55.117011 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 10:20:55.117072 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 10:20:55.117354 (+ 282us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 10:20:55.117355 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 10:20:55.117356 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 10:20:55.117383 (+ 27us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:20:55.117384 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:20:55.118275 (+ 891us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:20:55.118276 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:20:55.119482 (+ 1206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:20:55.119484 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:20:55.120478 (+ 994us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:20:55.120479 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:20:55.120549 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:20:55.120553 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 10:20:55.120598 (+ 45us) client_negotiation.cc:240] Negotiation successful
0619 10:20:55.120638 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":406,"thread_start_us":103,"threads_started":1}
I20260619 10:20:55.123067 19838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:20:55.116982 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33804 (local address 127.0.206.66:40061)
0619 10:20:55.117186 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0619 10:20:55.117190 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 10:20:55.117199 (+ 9us) server_negotiation.cc:407] Connection header received
0619 10:20:55.117242 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:20:55.117244 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:20:55.117260 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:20:55.117352 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 10:20:55.117434 (+ 82us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:20:55.118186 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:20:55.119572 (+ 1386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:20:55.120271 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:20:55.120340 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:20:55.120441 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0619 10:20:55.122942 (+ 2501us) server_negotiation.cc:299] Negotiation successful
0619 10:20:55.122988 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":172,"thread_start_us":95,"threads_started":1}
I20260619 10:20:55.123677 19628 tablet_service.cc:1467] Processing CreateTablet for tablet 1fa4f0ebcc36456f9ee4fd729f7d21f7 (DEFAULT_TABLE table=test-table [id=e3d98baab1514a008af419cf84d532d6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 10:20:55.123983 19628 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1fa4f0ebcc36456f9ee4fd729f7d21f7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 10:20:55.124579 19839 tablet_bootstrap.cc:492] T 1fa4f0ebcc36456f9ee4fd729f7d21f7 P 73b572f1a86d416691e3a7ef1cbc8f47: Bootstrap starting.
I20260619 10:20:55.124835 19840 tablet_bootstrap.cc:492] T 1fa4f0ebcc36456f9ee4fd729f7d21f7 P 6d841a2baa274687a27761a5c8eb77bc: Bootstrap starting.
I20260619 10:20:55.125368 19839 tablet_bootstrap.cc:654] T 1fa4f0ebcc36456f9ee4fd729f7d21f7 P 73b572f1a86d416691e3a7ef1cbc8f47: Neither blocks nor log segments found. Creating new log.
I20260619 10:20:55.125367 19840 tablet_bootstrap.cc:654] T 1fa4f0ebcc36456f9ee4fd729f7d21f7 P 6d841a2baa274687a27761a5c8eb77bc: Neither blocks nor log segments found. Creating new log.
I20260619 10:20:55.125677 19840 log.cc:824] T 1fa4f0ebcc36456f9ee4fd729f7d21f7 P 6d841a2baa274687a27761a5c8eb77bc: Log is configured to *not* fsync() on all Append() calls
I20260619 10:20:55.125677 19839 log.cc:824] T 1fa4f0ebcc36456f9ee4fd729f7d21f7 P 73b572f1a86d416691e3a7ef1cbc8f47: Log is configured to *not* fsync() on all Append() calls
I20260619 10:20:55.126408 19839 tablet_bootstrap.cc:492] T 1fa4f0ebcc36456f9ee4fd729f7d21f7 P 73b572f1a86d416691e3a7ef1cbc8f47: No bootstrap required, opened a new log
I20260619 10:20:55.126484 19839 ts_tablet_manager.cc:1397] T 1fa4f0ebcc36456f9ee4fd729f7d21f7 P 73b572f1a86d416691e3a7ef1cbc8f47: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260619 10:20:55.126682 19840 tablet_bootstrap.cc:492] T 1fa4f0ebcc36456f9ee4fd729f7d21f7 P 6d841a2baa274687a27761a5c8eb77bc: No bootstrap required, opened a new log
I20260619 10:20:55.126686 19843 tablet_bootstrap.cc:492] T 1fa4f0ebcc36456f9ee4fd729f7d21f7 P 552783c35ce14f7dbb694cfe5fa15d9c: Bootstrap starting.
I20260619 10:20:55.126731 19840 ts_tablet_manager.cc:1397] T 1fa4f0ebcc36456f9ee4fd729f7d21f7 P 6d841a2baa274687a27761a5c8eb77bc: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260619 10:20:55.127337 19843 tablet_bootstrap.cc:654] T 1fa4f0ebcc36456f9ee4fd729f7d21f7 P 552783c35ce14f7dbb694cfe5fa15d9c: Neither blocks nor log segments found. Creating new log.
I20260619 10:20:55.127636 19843 log.cc:824] T 1fa4f0ebcc36456f9ee4fd729f7d21f7 P 552783c35ce14f7dbb694cfe5fa15d9c: Log is configured to *not* fsync() on all Append() calls
I20260619 10:20:55.128258 19843 tablet_bootstrap.cc:492] T 1fa4f0ebcc36456f9ee4fd729f7d21f7 P 552783c35ce14f7dbb694cfe5fa15d9c: No bootstrap required, opened a new log
I20260619 10:20:55.128350 19843 ts_tablet_manager.cc:1397] T 1fa4f0ebcc36456f9ee4fd729f7d21f7 P 552783c35ce14f7dbb694cfe5fa15d9c: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260619 10:20:55.129132 19839 raft_consensus.cc:348] T 1fa4f0ebcc36456f9ee4fd729f7d21f7 P 73b572f1a86d416691e3a7ef1cbc8f47 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d841a2baa274687a27761a5c8eb77bc" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 37791 } } peers { permanent_uuid: "73b572f1a86d416691e3a7ef1cbc8f47" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 43293 } } peers { permanent_uuid: "552783c35ce14f7dbb694cfe5fa15d9c" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 40061 } }
I20260619 10:20:55.129148 19840 raft_consensus.cc:348] T 1fa4f0ebcc36456f9ee4fd729f7d21f7 P 6d841a2baa274687a27761a5c8eb77bc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d841a2baa274687a27761a5c8eb77bc" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 37791 } } peers { permanent_uuid: "73b572f1a86d416691e3a7ef1cbc8f47" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 43293 } } peers { permanent_uuid: "552783c35ce14f7dbb694cfe5fa15d9c" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 40061 } }
I20260619 10:20:55.129297 19840 raft_consensus.cc:374] T 1fa4f0ebcc36456f9ee4fd729f7d21f7 P 6d841a2baa274687a27761a5c8eb77bc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 10:20:55.129297 19839 raft_consensus.cc:374] T 1fa4f0ebcc36456f9ee4fd729f7d21f7 P 73b572f1a86d416691e3a7ef1cbc8f47 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 10:20:55.129331 19839 raft_consensus.cc:729] T 1fa4f0ebcc36456f9ee4fd729f7d21f7 P 73b572f1a86d416691e3a7ef1cbc8f47 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 73b572f1a86d416691e3a7ef1cbc8f47, State: Initialized, Role: FOLLOWER
I20260619 10:20:55.129331 19840 raft_consensus.cc:729] T 1fa4f0ebcc36456f9ee4fd729f7d21f7 P 6d841a2baa274687a27761a5c8eb77bc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6d841a2baa274687a27761a5c8eb77bc, State: Initialized, Role: FOLLOWER
I20260619 10:20:55.129465 19840 consensus_queue.cc:260] T 1fa4f0ebcc36456f9ee4fd729f7d21f7 P 6d841a2baa274687a27761a5c8eb77bc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d841a2baa274687a27761a5c8eb77bc" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 37791 } } peers { permanent_uuid: "73b572f1a86d416691e3a7ef1cbc8f47" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 43293 } } peers { permanent_uuid: "552783c35ce14f7dbb694cfe5fa15d9c" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 40061 } }
I20260619 10:20:55.129465 19839 consensus_queue.cc:260] T 1fa4f0ebcc36456f9ee4fd729f7d21f7 P 73b572f1a86d416691e3a7ef1cbc8f47 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d841a2baa274687a27761a5c8eb77bc" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 37791 } } peers { permanent_uuid: "73b572f1a86d416691e3a7ef1cbc8f47" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 43293 } } peers { permanent_uuid: "552783c35ce14f7dbb694cfe5fa15d9c" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 40061 } }
I20260619 10:20:55.129716 19840 ts_tablet_manager.cc:1428] T 1fa4f0ebcc36456f9ee4fd729f7d21f7 P 6d841a2baa274687a27761a5c8eb77bc: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260619 10:20:55.129716 19839 ts_tablet_manager.cc:1428] T 1fa4f0ebcc36456f9ee4fd729f7d21f7 P 73b572f1a86d416691e3a7ef1cbc8f47: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260619 10:20:55.129846 19563 heartbeater.cc:503] Master 127.0.206.126:38693 was elected leader, sending a full tablet report...
I20260619 10:20:55.129855 19825 heartbeater.cc:503] Master 127.0.206.126:38693 was elected leader, sending a full tablet report...
I20260619 10:20:55.131070 19843 raft_consensus.cc:348] T 1fa4f0ebcc36456f9ee4fd729f7d21f7 P 552783c35ce14f7dbb694cfe5fa15d9c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d841a2baa274687a27761a5c8eb77bc" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 37791 } } peers { permanent_uuid: "73b572f1a86d416691e3a7ef1cbc8f47" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 43293 } } peers { permanent_uuid: "552783c35ce14f7dbb694cfe5fa15d9c" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 40061 } }
I20260619 10:20:55.131244 19843 raft_consensus.cc:374] T 1fa4f0ebcc36456f9ee4fd729f7d21f7 P 552783c35ce14f7dbb694cfe5fa15d9c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 10:20:55.131287 19843 raft_consensus.cc:729] T 1fa4f0ebcc36456f9ee4fd729f7d21f7 P 552783c35ce14f7dbb694cfe5fa15d9c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 552783c35ce14f7dbb694cfe5fa15d9c, State: Initialized, Role: FOLLOWER
I20260619 10:20:55.131397 19843 consensus_queue.cc:260] T 1fa4f0ebcc36456f9ee4fd729f7d21f7 P 552783c35ce14f7dbb694cfe5fa15d9c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d841a2baa274687a27761a5c8eb77bc" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 37791 } } peers { permanent_uuid: "73b572f1a86d416691e3a7ef1cbc8f47" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 43293 } } peers { permanent_uuid: "552783c35ce14f7dbb694cfe5fa15d9c" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 40061 } }
I20260619 10:20:55.131633 19843 ts_tablet_manager.cc:1428] T 1fa4f0ebcc36456f9ee4fd729f7d21f7 P 552783c35ce14f7dbb694cfe5fa15d9c: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260619 10:20:55.131680 19694 heartbeater.cc:503] Master 127.0.206.126:38693 was elected leader, sending a full tablet report...
W20260619 10:20:55.228418 19695 tablet.cc:2369] T 1fa4f0ebcc36456f9ee4fd729f7d21f7 P 552783c35ce14f7dbb694cfe5fa15d9c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 10:20:55.271118 19846 raft_consensus.cc:482] T 1fa4f0ebcc36456f9ee4fd729f7d21f7 P 6d841a2baa274687a27761a5c8eb77bc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 10:20:55.271260 19846 raft_consensus.cc:504] T 1fa4f0ebcc36456f9ee4fd729f7d21f7 P 6d841a2baa274687a27761a5c8eb77bc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d841a2baa274687a27761a5c8eb77bc" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 37791 } } peers { permanent_uuid: "73b572f1a86d416691e3a7ef1cbc8f47" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 43293 } } peers { permanent_uuid: "552783c35ce14f7dbb694cfe5fa15d9c" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 40061 } }
I20260619 10:20:55.271626 19846 leader_election.cc:288] T 1fa4f0ebcc36456f9ee4fd729f7d21f7 P 6d841a2baa274687a27761a5c8eb77bc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 73b572f1a86d416691e3a7ef1cbc8f47 (127.0.206.65:43293), 552783c35ce14f7dbb694cfe5fa15d9c (127.0.206.66:40061)
I20260619 10:20:55.275493 19836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:20:55.271894 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.67:55131 (local address 127.0.206.65:43293)
0619 10:20:55.271940 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0619 10:20:55.271942 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 10:20:55.271952 (+ 10us) server_negotiation.cc:407] Connection header received
0619 10:20:55.271981 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:20:55.271982 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:20:55.272000 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:20:55.272051 (+ 51us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 10:20:55.272243 (+ 192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:20:55.273041 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:20:55.274532 (+ 1491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:20:55.275121 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:20:55.275196 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:20:55.275204 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0619 10:20:55.275385 (+ 181us) server_negotiation.cc:299] Negotiation successful
0619 10:20:55.275425 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260619 10:20:55.275496 19838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:20:55.271913 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.67:57505 (local address 127.0.206.66:40061)
0619 10:20:55.271957 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0619 10:20:55.271959 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 10:20:55.272085 (+ 126us) server_negotiation.cc:407] Connection header received
0619 10:20:55.272119 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:20:55.272119 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:20:55.272133 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:20:55.272174 (+ 41us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 10:20:55.272320 (+ 146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:20:55.273047 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:20:55.274533 (+ 1486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:20:55.275126 (+ 593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:20:55.275196 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:20:55.275204 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0619 10:20:55.275399 (+ 195us) server_negotiation.cc:299] Negotiation successful
0619 10:20:55.275437 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260619 10:20:55.275485 19827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:20:55.271800 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.65:43293 (local address 127.0.206.67:55131)
0619 10:20:55.271861 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0619 10:20:55.271873 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 10:20:55.271914 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 10:20:55.272074 (+ 160us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 10:20:55.272075 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 10:20:55.272076 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 10:20:55.272143 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:20:55.272146 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:20:55.273155 (+ 1009us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:20:55.273156 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:20:55.274441 (+ 1285us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:20:55.274444 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:20:55.275196 (+ 752us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:20:55.275197 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:20:55.275278 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:20:55.275284 (+ 6us) client_negotiation.cc:769] Sending connection context
0619 10:20:55.275343 (+ 59us) client_negotiation.cc:240] Negotiation successful
0619 10:20:55.275393 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":44,"mutex_wait_us":15}
I20260619 10:20:55.275492 19848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:20:55.271800 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.66:40061 (local address 127.0.206.67:57505)
0619 10:20:55.272002 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0619 10:20:55.272009 (+ 7us) client_negotiation.cc:174] Beginning negotiation
0619 10:20:55.272073 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 10:20:55.272193 (+ 120us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 10:20:55.272193 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 10:20:55.272195 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 10:20:55.272250 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:20:55.272253 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:20:55.273155 (+ 902us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:20:55.273156 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:20:55.274446 (+ 1290us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:20:55.274449 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:20:55.275196 (+ 747us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:20:55.275197 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:20:55.275278 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:20:55.275284 (+ 6us) client_negotiation.cc:769] Sending connection context
0619 10:20:55.275384 (+ 100us) client_negotiation.cc:240] Negotiation successful
0619 10:20:55.275411 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":174,"mutex_wait_us":104,"thread_start_us":107,"threads_started":1}
I20260619 10:20:55.275887 19648 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1fa4f0ebcc36456f9ee4fd729f7d21f7" candidate_uuid: "6d841a2baa274687a27761a5c8eb77bc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "552783c35ce14f7dbb694cfe5fa15d9c" is_pre_election: true
I20260619 10:20:55.275877 19517 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1fa4f0ebcc36456f9ee4fd729f7d21f7" candidate_uuid: "6d841a2baa274687a27761a5c8eb77bc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "73b572f1a86d416691e3a7ef1cbc8f47" is_pre_election: true
I20260619 10:20:55.276033 19517 raft_consensus.cc:2454] T 1fa4f0ebcc36456f9ee4fd729f7d21f7 P 73b572f1a86d416691e3a7ef1cbc8f47 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6d841a2baa274687a27761a5c8eb77bc in term 0.
I20260619 10:20:55.276033 19648 raft_consensus.cc:2454] T 1fa4f0ebcc36456f9ee4fd729f7d21f7 P 552783c35ce14f7dbb694cfe5fa15d9c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6d841a2baa274687a27761a5c8eb77bc in term 0.
I20260619 10:20:55.276268 19712 leader_election.cc:302] T 1fa4f0ebcc36456f9ee4fd729f7d21f7 P 6d841a2baa274687a27761a5c8eb77bc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 552783c35ce14f7dbb694cfe5fa15d9c, 6d841a2baa274687a27761a5c8eb77bc; no voters:
I20260619 10:20:55.276408 19846 raft_consensus.cc:2781] T 1fa4f0ebcc36456f9ee4fd729f7d21f7 P 6d841a2baa274687a27761a5c8eb77bc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 10:20:55.276492 19846 raft_consensus.cc:482] T 1fa4f0ebcc36456f9ee4fd729f7d21f7 P 6d841a2baa274687a27761a5c8eb77bc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 10:20:55.276528 19846 raft_consensus.cc:3037] T 1fa4f0ebcc36456f9ee4fd729f7d21f7 P 6d841a2baa274687a27761a5c8eb77bc [term 0 FOLLOWER]: Advancing to term 1
I20260619 10:20:55.277243 19846 raft_consensus.cc:504] T 1fa4f0ebcc36456f9ee4fd729f7d21f7 P 6d841a2baa274687a27761a5c8eb77bc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d841a2baa274687a27761a5c8eb77bc" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 37791 } } peers { permanent_uuid: "73b572f1a86d416691e3a7ef1cbc8f47" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 43293 } } peers { permanent_uuid: "552783c35ce14f7dbb694cfe5fa15d9c" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 40061 } }
I20260619 10:20:55.277406 19846 leader_election.cc:288] T 1fa4f0ebcc36456f9ee4fd729f7d21f7 P 6d841a2baa274687a27761a5c8eb77bc [CANDIDATE]: Term 1 election: Requested vote from peers 73b572f1a86d416691e3a7ef1cbc8f47 (127.0.206.65:43293), 552783c35ce14f7dbb694cfe5fa15d9c (127.0.206.66:40061)
I20260619 10:20:55.277539 19517 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1fa4f0ebcc36456f9ee4fd729f7d21f7" candidate_uuid: "6d841a2baa274687a27761a5c8eb77bc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "73b572f1a86d416691e3a7ef1cbc8f47"
I20260619 10:20:55.277559 19648 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1fa4f0ebcc36456f9ee4fd729f7d21f7" candidate_uuid: "6d841a2baa274687a27761a5c8eb77bc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "552783c35ce14f7dbb694cfe5fa15d9c"
I20260619 10:20:55.277623 19517 raft_consensus.cc:3037] T 1fa4f0ebcc36456f9ee4fd729f7d21f7 P 73b572f1a86d416691e3a7ef1cbc8f47 [term 0 FOLLOWER]: Advancing to term 1
I20260619 10:20:55.277623 19648 raft_consensus.cc:3037] T 1fa4f0ebcc36456f9ee4fd729f7d21f7 P 552783c35ce14f7dbb694cfe5fa15d9c [term 0 FOLLOWER]: Advancing to term 1
I20260619 10:20:55.278508 19648 raft_consensus.cc:2454] T 1fa4f0ebcc36456f9ee4fd729f7d21f7 P 552783c35ce14f7dbb694cfe5fa15d9c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6d841a2baa274687a27761a5c8eb77bc in term 1.
I20260619 10:20:55.278508 19517 raft_consensus.cc:2454] T 1fa4f0ebcc36456f9ee4fd729f7d21f7 P 73b572f1a86d416691e3a7ef1cbc8f47 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6d841a2baa274687a27761a5c8eb77bc in term 1.
I20260619 10:20:55.278704 19715 leader_election.cc:302] T 1fa4f0ebcc36456f9ee4fd729f7d21f7 P 6d841a2baa274687a27761a5c8eb77bc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6d841a2baa274687a27761a5c8eb77bc, 73b572f1a86d416691e3a7ef1cbc8f47; no voters:
I20260619 10:20:55.278812 19846 raft_consensus.cc:2781] T 1fa4f0ebcc36456f9ee4fd729f7d21f7 P 6d841a2baa274687a27761a5c8eb77bc [term 1 FOLLOWER]: Leader election won for term 1
I20260619 10:20:55.278944 19846 raft_consensus.cc:686] T 1fa4f0ebcc36456f9ee4fd729f7d21f7 P 6d841a2baa274687a27761a5c8eb77bc [term 1 LEADER]: Becoming Leader. State: Replica: 6d841a2baa274687a27761a5c8eb77bc, State: Running, Role: LEADER
I20260619 10:20:55.279011 19846 consensus_queue.cc:237] T 1fa4f0ebcc36456f9ee4fd729f7d21f7 P 6d841a2baa274687a27761a5c8eb77bc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d841a2baa274687a27761a5c8eb77bc" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 37791 } } peers { permanent_uuid: "73b572f1a86d416691e3a7ef1cbc8f47" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 43293 } } peers { permanent_uuid: "552783c35ce14f7dbb694cfe5fa15d9c" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 40061 } }
I20260619 10:20:55.279812 19374 catalog_manager.cc:5495] T 1fa4f0ebcc36456f9ee4fd729f7d21f7 P 6d841a2baa274687a27761a5c8eb77bc reported cstate change: term changed from 0 to 1, leader changed from <none> to 6d841a2baa274687a27761a5c8eb77bc (127.0.206.67). New cstate: current_term: 1 leader_uuid: "6d841a2baa274687a27761a5c8eb77bc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d841a2baa274687a27761a5c8eb77bc" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 37791 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "73b572f1a86d416691e3a7ef1cbc8f47" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 43293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "552783c35ce14f7dbb694cfe5fa15d9c" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 40061 } health_report { overall_health: UNKNOWN } } }
I20260619 10:20:55.302836 19837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:20:55.300699 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52148 (local address 127.0.206.67:37791)
0619 10:20:55.300742 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0619 10:20:55.300743 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 10:20:55.300750 (+ 7us) server_negotiation.cc:407] Connection header received
0619 10:20:55.300803 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:20:55.300804 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:20:55.300820 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:20:55.300879 (+ 59us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 10:20:55.300985 (+ 106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:20:55.301455 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:20:55.302147 (+ 692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:20:55.302333 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:20:55.302380 (+ 47us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:20:55.302502 (+ 122us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 10:20:55.302580 (+ 78us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 10:20:55.302709 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0619 10:20:55.302759 (+ 50us) server_negotiation.cc:299] Negotiation successful
0619 10:20:55.302766 (+ 7us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260619 10:20:55.304497 19517 raft_consensus.cc:1261] T 1fa4f0ebcc36456f9ee4fd729f7d21f7 P 73b572f1a86d416691e3a7ef1cbc8f47 [term 1 FOLLOWER]: Refusing update from remote peer 6d841a2baa274687a27761a5c8eb77bc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 10:20:55.304497 19648 raft_consensus.cc:1261] T 1fa4f0ebcc36456f9ee4fd729f7d21f7 P 552783c35ce14f7dbb694cfe5fa15d9c [term 1 FOLLOWER]: Refusing update from remote peer 6d841a2baa274687a27761a5c8eb77bc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 10:20:55.304854 19849 consensus_queue.cc:1035] T 1fa4f0ebcc36456f9ee4fd729f7d21f7 P 6d841a2baa274687a27761a5c8eb77bc [LEADER]: Connected to new peer: Peer: permanent_uuid: "73b572f1a86d416691e3a7ef1cbc8f47" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 43293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 10:20:55.304981 19846 consensus_queue.cc:1035] T 1fa4f0ebcc36456f9ee4fd729f7d21f7 P 6d841a2baa274687a27761a5c8eb77bc [LEADER]: Connected to new peer: Peer: permanent_uuid: "552783c35ce14f7dbb694cfe5fa15d9c" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 40061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 10:20:55.309998 19375 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:54116:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 10:20:55.310102 19375 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:54116:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 10:20:55.310844 19375 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3cddd035221d435f91f59c4d11a1647f: Sending DeleteTablet for 3 replicas of tablet 1fa4f0ebcc36456f9ee4fd729f7d21f7
I20260619 10:20:55.311209 19759 tablet_service.cc:1514] Processing DeleteTablet for tablet 1fa4f0ebcc36456f9ee4fd729f7d21f7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 10:20:55 UTC) from {username='slave'} at 127.0.0.1:52136
I20260619 10:20:55.311214 19497 tablet_service.cc:1514] Processing DeleteTablet for tablet 1fa4f0ebcc36456f9ee4fd729f7d21f7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 10:20:55 UTC) from {username='slave'} at 127.0.0.1:36136
I20260619 10:20:55.311232 19628 tablet_service.cc:1514] Processing DeleteTablet for tablet 1fa4f0ebcc36456f9ee4fd729f7d21f7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 10:20:55 UTC) from {username='slave'} at 127.0.0.1:33804
I20260619 10:20:55.311414 19862 tablet_replica.cc:330] T 1fa4f0ebcc36456f9ee4fd729f7d21f7 P 6d841a2baa274687a27761a5c8eb77bc: stopping tablet replica
I20260619 10:20:55.311473 19861 tablet_replica.cc:330] T 1fa4f0ebcc36456f9ee4fd729f7d21f7 P 73b572f1a86d416691e3a7ef1cbc8f47: stopping tablet replica
I20260619 10:20:55.311509 19862 raft_consensus.cc:2229] T 1fa4f0ebcc36456f9ee4fd729f7d21f7 P 6d841a2baa274687a27761a5c8eb77bc [term 1 LEADER]: Raft consensus shutting down.
I20260619 10:20:55.311542 19861 raft_consensus.cc:2229] T 1fa4f0ebcc36456f9ee4fd729f7d21f7 P 73b572f1a86d416691e3a7ef1cbc8f47 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 10:20:55.311586 19862 raft_consensus.cc:2258] T 1fa4f0ebcc36456f9ee4fd729f7d21f7 P 6d841a2baa274687a27761a5c8eb77bc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 10:20:55.311627 19861 raft_consensus.cc:2258] T 1fa4f0ebcc36456f9ee4fd729f7d21f7 P 73b572f1a86d416691e3a7ef1cbc8f47 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 10:20:55.311635 825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTU9OP6/build/release/bin/kudu with pid 19435
I20260619 10:20:55.311838 19863 tablet_replica.cc:330] T 1fa4f0ebcc36456f9ee4fd729f7d21f7 P 552783c35ce14f7dbb694cfe5fa15d9c: stopping tablet replica
I20260619 10:20:55.311936 19863 raft_consensus.cc:2229] T 1fa4f0ebcc36456f9ee4fd729f7d21f7 P 552783c35ce14f7dbb694cfe5fa15d9c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 10:20:55.312011 19863 raft_consensus.cc:2258] T 1fa4f0ebcc36456f9ee4fd729f7d21f7 P 552783c35ce14f7dbb694cfe5fa15d9c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 10:20:55.313082 19862 ts_tablet_manager.cc:1905] T 1fa4f0ebcc36456f9ee4fd729f7d21f7 P 6d841a2baa274687a27761a5c8eb77bc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 10:20:55.314399 19862 ts_tablet_manager.cc:1918] T 1fa4f0ebcc36456f9ee4fd729f7d21f7 P 6d841a2baa274687a27761a5c8eb77bc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 10:20:55.314457 19862 log.cc:1196] T 1fa4f0ebcc36456f9ee4fd729f7d21f7 P 6d841a2baa274687a27761a5c8eb77bc: Deleting WAL directory at /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781864224577058-825-0/minicluster-data/ts-2/wal/wals/1fa4f0ebcc36456f9ee4fd729f7d21f7
I20260619 10:20:55.314492 19863 ts_tablet_manager.cc:1905] T 1fa4f0ebcc36456f9ee4fd729f7d21f7 P 552783c35ce14f7dbb694cfe5fa15d9c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 10:20:55.314742 19862 ts_tablet_manager.cc:1939] T 1fa4f0ebcc36456f9ee4fd729f7d21f7 P 6d841a2baa274687a27761a5c8eb77bc: Deleting consensus metadata
I20260619 10:20:55.315084 19360 catalog_manager.cc:4841] TS 6d841a2baa274687a27761a5c8eb77bc (127.0.206.67:37791): tablet 1fa4f0ebcc36456f9ee4fd729f7d21f7 (table test-table [id=e3d98baab1514a008af419cf84d532d6]) successfully deleted
I20260619 10:20:55.315826 19863 ts_tablet_manager.cc:1918] T 1fa4f0ebcc36456f9ee4fd729f7d21f7 P 552783c35ce14f7dbb694cfe5fa15d9c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 10:20:55.315871 19863 log.cc:1196] T 1fa4f0ebcc36456f9ee4fd729f7d21f7 P 552783c35ce14f7dbb694cfe5fa15d9c: Deleting WAL directory at /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781864224577058-825-0/minicluster-data/ts-1/wal/wals/1fa4f0ebcc36456f9ee4fd729f7d21f7
I20260619 10:20:55.316039 19863 ts_tablet_manager.cc:1939] T 1fa4f0ebcc36456f9ee4fd729f7d21f7 P 552783c35ce14f7dbb694cfe5fa15d9c: Deleting consensus metadata
I20260619 10:20:55.316339 19359 catalog_manager.cc:4841] TS 552783c35ce14f7dbb694cfe5fa15d9c (127.0.206.66:40061): tablet 1fa4f0ebcc36456f9ee4fd729f7d21f7 (table test-table [id=e3d98baab1514a008af419cf84d532d6]) successfully deleted
W20260619 10:20:55.320279 19362 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.0.206.65:43293 (error 108)
I20260619 10:20:55.320446 825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTU9OP6/build/release/bin/kudu with pid 19566
I20260619 10:20:55.321102 19834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:20:55.320593 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.65:43293 (local address 127.0.0.1:36140)
0619 10:20:55.321029 (+ 436us) negotiation.cc:107] Waiting for socket to connect
0619 10:20:55.321064 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.65:43293: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":417}
W20260619 10:20:55.322440 19362 catalog_manager.cc:4584] TS 73b572f1a86d416691e3a7ef1cbc8f47 (127.0.206.65:43293): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 1fa4f0ebcc36456f9ee4fd729f7d21f7: Network error: Client connection negotiation failed: client connection to 127.0.206.65:43293: connect: Connection refused (error 111)
I20260619 10:20:55.325529 825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTU9OP6/build/release/bin/kudu with pid 19697
I20260619 10:20:55.329607 825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTU9OP6/build/release/bin/kudu with pid 19344
2026-06-19T10:20:55Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (802 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781864224577058-825-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 10:20:55 dist-test-slave-6pt2 krb5kdc[19867](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 10:20:55 dist-test-slave-6pt2 krb5kdc[19867](info): set up 2 sockets
Jun 19 10:20:55 dist-test-slave-6pt2 krb5kdc[19867](info): commencing operation
krb5kdc: starting...
W20260619 10:20:56.335110 825 mini_kdc.cc:121] Time spent starting KDC: real 0.988s user 0.003s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781864224577058-825-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781864224577058-825-0/krb5kdc/test-user.keytab.
Jun 19 10:20:56 dist-test-slave-6pt2 krb5kdc[19867](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864456, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T10:20:56Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T10:20:56Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.206.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.206.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.206.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781864224577058-825-0/krb5kdc/kudu_127.0.206.126.keytab.
Entry for principal kudu/127.0.206.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781864224577058-825-0/krb5kdc/kudu_127.0.206.126.keytab.
I20260619 10:20:56.441118 825 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTU9OP6/build/release/bin/kudu
/tmp/dist-test-taskTU9OP6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781864224577058-825-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781864224577058-825-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781864224577058-825-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781864224577058-825-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.206.126:42821
--webserver_interface=127.0.206.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781864224577058-825-0/krb5kdc/kudu_127.0.206.126.keytab
--principal=kudu/127.0.206.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.206.84:36119
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.206.126:42821
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781864224577058-825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781864224577058-825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781864224577058-825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 10:20:56.516996 19880 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 10:20:56.517189 19880 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 10:20:56.517208 19880 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 10:20:56.518275 19880 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 10:20:56.518309 19880 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 10:20:56.518323 19880 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 10:20:56.518337 19880 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 10:20:56.518347 19880 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 10:20:56.519701 19880 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.206.84:36119
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781864224577058-825-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781864224577058-825-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.206.126:42821
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781864224577058-825-0/krb5kdc/kudu_127.0.206.126.keytab
--principal=kudu/127.0.206.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.206.126:42821
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781864224577058-825-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.206.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19880
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781864224577058-825-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 09:32:44 UTC on ccycloud.quasar-gbtldp.root.comops.site
build id 4455440
I20260619 10:20:56.519866 19880 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 10:20:56.520008 19880 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260619 10:20:56.524580 19880 server_base.cc:1034] running on GCE node
W20260619 10:20:56.524554 19885 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 10:20:56.524554 19888 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 10:20:56.524545 19886 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 10:20:56.524953 19880 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 10:20:56.525239 19880 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 10:20:56.526382 19880 hybrid_clock.cc:648] HybridClock initialized: now 1781864456526362 us; error 37 us; skew 500 ppm
Jun 19 10:20:56 dist-test-slave-6pt2 krb5kdc[19867](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864456, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 10:20:56.528777 19880 init.cc:377] Logged in from keytab as kudu/127.0.206.126@KRBTEST.COM (short username kudu)
I20260619 10:20:56.529107 19880 webserver.cc:460] Webserver started at http://127.0.206.126:35763/ using document root <none> and password file <none>
I20260619 10:20:56.529273 19880 fs_manager.cc:359] Metadata directory not provided
I20260619 10:20:56.529310 19880 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 10:20:56.529392 19880 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 10:20:56.530280 19880 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781864224577058-825-0/minicluster-data/master-0/data/instance:
uuid: "03d2885f88484238ac53707a901ae3f7"
format_stamp: "Formatted at 2026-06-19 10:20:56 on dist-test-slave-6pt2"
server_key: "a05ae697430c7e130ae90c20e2f5aa7d"
server_key_iv: "9dec5464ff52fd84ef93c0fe86956145"
server_key_version: "encryptionkey@0"
I20260619 10:20:56.530562 19880 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781864224577058-825-0/minicluster-data/master-0/wal/instance:
uuid: "03d2885f88484238ac53707a901ae3f7"
format_stamp: "Formatted at 2026-06-19 10:20:56 on dist-test-slave-6pt2"
server_key: "a05ae697430c7e130ae90c20e2f5aa7d"
server_key_iv: "9dec5464ff52fd84ef93c0fe86956145"
server_key_version: "encryptionkey@0"
I20260619 10:20:56.531812 19880 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 10:20:56.532423 19895 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 10:20:56.532565 19880 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 10:20:56.532644 19880 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781864224577058-825-0/minicluster-data/master-0/data,/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781864224577058-825-0/minicluster-data/master-0/wal
uuid: "03d2885f88484238ac53707a901ae3f7"
format_stamp: "Formatted at 2026-06-19 10:20:56 on dist-test-slave-6pt2"
server_key: "a05ae697430c7e130ae90c20e2f5aa7d"
server_key_iv: "9dec5464ff52fd84ef93c0fe86956145"
server_key_version: "encryptionkey@0"
I20260619 10:20:56.532693 19880 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781864224577058-825-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781864224577058-825-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781864224577058-825-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 10:20:56.550570 19880 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 10:20:56.552197 19880 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 10:20:56.552331 19880 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 10:20:56.556026 19880 rpc_server.cc:307] RPC server started. Bound to: 127.0.206.126:42821
I20260619 10:20:56.556092 19947 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.206.126:42821 every 8 connection(s)
I20260619 10:20:56.556401 19880 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781864224577058-825-0/minicluster-data/master-0/data/info.pb
I20260619 10:20:56.557200 19948 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 10:20:56.559353 19948 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 03d2885f88484238ac53707a901ae3f7: Bootstrap starting.
I20260619 10:20:56.559849 19948 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 03d2885f88484238ac53707a901ae3f7: Neither blocks nor log segments found. Creating new log.
I20260619 10:20:56.560073 19948 log.cc:824] T 00000000000000000000000000000000 P 03d2885f88484238ac53707a901ae3f7: Log is configured to *not* fsync() on all Append() calls
I20260619 10:20:56.560665 19948 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 03d2885f88484238ac53707a901ae3f7: No bootstrap required, opened a new log
I20260619 10:20:56.562069 19948 raft_consensus.cc:348] T 00000000000000000000000000000000 P 03d2885f88484238ac53707a901ae3f7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "03d2885f88484238ac53707a901ae3f7" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 42821 } }
I20260619 10:20:56.562188 19948 raft_consensus.cc:374] T 00000000000000000000000000000000 P 03d2885f88484238ac53707a901ae3f7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 10:20:56.562209 19948 raft_consensus.cc:729] T 00000000000000000000000000000000 P 03d2885f88484238ac53707a901ae3f7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 03d2885f88484238ac53707a901ae3f7, State: Initialized, Role: FOLLOWER
I20260619 10:20:56.562302 19948 consensus_queue.cc:260] T 00000000000000000000000000000000 P 03d2885f88484238ac53707a901ae3f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "03d2885f88484238ac53707a901ae3f7" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 42821 } }
I20260619 10:20:56.562381 19948 raft_consensus.cc:388] T 00000000000000000000000000000000 P 03d2885f88484238ac53707a901ae3f7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 10:20:56.562414 19948 raft_consensus.cc:482] T 00000000000000000000000000000000 P 03d2885f88484238ac53707a901ae3f7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 10:20:56.562471 19948 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 03d2885f88484238ac53707a901ae3f7 [term 0 FOLLOWER]: Advancing to term 1
I20260619 10:20:56.562985 19948 raft_consensus.cc:504] T 00000000000000000000000000000000 P 03d2885f88484238ac53707a901ae3f7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "03d2885f88484238ac53707a901ae3f7" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 42821 } }
I20260619 10:20:56.563097 19948 leader_election.cc:302] T 00000000000000000000000000000000 P 03d2885f88484238ac53707a901ae3f7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 03d2885f88484238ac53707a901ae3f7; no voters:
I20260619 10:20:56.563277 19948 leader_election.cc:288] T 00000000000000000000000000000000 P 03d2885f88484238ac53707a901ae3f7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 10:20:56.563314 19951 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 03d2885f88484238ac53707a901ae3f7 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 10:20:56.563506 19951 raft_consensus.cc:686] T 00000000000000000000000000000000 P 03d2885f88484238ac53707a901ae3f7 [term 1 LEADER]: Becoming Leader. State: Replica: 03d2885f88484238ac53707a901ae3f7, State: Running, Role: LEADER
I20260619 10:20:56.563537 19948 sys_catalog.cc:564] T 00000000000000000000000000000000 P 03d2885f88484238ac53707a901ae3f7 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 10:20:56.563586 19951 consensus_queue.cc:237] T 00000000000000000000000000000000 P 03d2885f88484238ac53707a901ae3f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "03d2885f88484238ac53707a901ae3f7" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 42821 } }
I20260619 10:20:56.563899 19953 sys_catalog.cc:455] T 00000000000000000000000000000000 P 03d2885f88484238ac53707a901ae3f7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 03d2885f88484238ac53707a901ae3f7. Latest consensus state: current_term: 1 leader_uuid: "03d2885f88484238ac53707a901ae3f7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "03d2885f88484238ac53707a901ae3f7" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 42821 } } }
I20260619 10:20:56.564008 19952 sys_catalog.cc:455] T 00000000000000000000000000000000 P 03d2885f88484238ac53707a901ae3f7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "03d2885f88484238ac53707a901ae3f7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "03d2885f88484238ac53707a901ae3f7" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 42821 } } }
I20260619 10:20:56.564062 19953 sys_catalog.cc:458] T 00000000000000000000000000000000 P 03d2885f88484238ac53707a901ae3f7 [sys.catalog]: This master's current role is: LEADER
I20260619 10:20:56.564091 19952 sys_catalog.cc:458] T 00000000000000000000000000000000 P 03d2885f88484238ac53707a901ae3f7 [sys.catalog]: This master's current role is: LEADER
I20260619 10:20:56.564450 19960 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 10:20:56.564929 825 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTU9OP6/build/release/bin/kudu as pid 19880
I20260619 10:20:56.565052 825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781864224577058-825-0/minicluster-data/master-0/wal/instance
I20260619 10:20:56.565240 825 external_mini_cluster.cc:1428] Setting key 8a70ccbd6926543920c3260ac8df8057
I20260619 10:20:56.565637 19960 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 10:20:56.567405 19960 catalog_manager.cc:1347] Generated new cluster ID: 0174264119534e28a7b72b75e2594823
I20260619 10:20:56.567456 19960 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Jun 19 10:20:56 dist-test-slave-6pt2 krb5kdc[19867](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864456, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.206.126@KRBTEST.COM
I20260619 10:20:56.573096 19960 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 10:20:56.573746 19960 catalog_manager.cc:1504] Loading token signing keys...
I20260619 10:20:56.575209 19970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:20:56.565837 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54600 (local address 127.0.206.126:42821)
0619 10:20:56.566401 (+ 564us) server_negotiation.cc:206] Beginning negotiation
0619 10:20:56.566410 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 10:20:56.566444 (+ 34us) server_negotiation.cc:407] Connection header received
0619 10:20:56.566560 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:20:56.566567 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:20:56.566761 (+ 194us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:20:56.566904 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:20:56.567565 (+ 661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:20:56.568475 (+ 910us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:20:56.569197 (+ 722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:20:56.569602 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:20:56.569717 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:20:56.571958 (+ 2241us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 10:20:56.571962 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 10:20:56.571962 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 10:20:56.572010 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0619 10:20:56.573686 (+ 1676us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:20:56.574066 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:20:56.574069 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:20:56.574069 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:20:56.574158 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:20:56.574361 (+ 203us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:20:56.574362 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:20:56.574362 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:20:56.574568 (+ 206us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 10:20:56.574782 (+ 214us) server_negotiation.cc:1035] Waiting for connection context
0619 10:20:56.574871 (+ 89us) server_negotiation.cc:299] Negotiation successful
0619 10:20:56.575023 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":522,"thread_start_us":74,"threads_started":1}
I20260619 10:20:56.587646 19960 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 03d2885f88484238ac53707a901ae3f7: Generated new TSK 0
I20260619 10:20:56.587785 19960 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.0.206.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.206.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.206.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781864224577058-825-0/krb5kdc/kudu_127.0.206.65.keytab.
Entry for principal kudu/127.0.206.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781864224577058-825-0/krb5kdc/kudu_127.0.206.65.keytab.
I20260619 10:20:56.618237 825 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTU9OP6/build/release/bin/kudu
/tmp/dist-test-taskTU9OP6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781864224577058-825-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781864224577058-825-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781864224577058-825-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781864224577058-825-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.206.65:0
--local_ip_for_outbound_sockets=127.0.206.65
--webserver_interface=127.0.206.65
--webserver_port=0
--tserver_master_addrs=127.0.206.126:42821
--keytab_file=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781864224577058-825-0/krb5kdc/kudu_127.0.206.65.keytab
--principal=kudu/127.0.206.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.206.84:36119
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781864224577058-825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781864224577058-825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781864224577058-825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 10:20:56.697072 19974 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 10:20:56.697244 19974 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 10:20:56.697266 19974 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 10:20:56.698472 19974 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 10:20:56.698521 19974 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 10:20:56.698570 19974 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.206.65
I20260619 10:20:56.700098 19974 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.206.84:36119
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781864224577058-825-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781864224577058-825-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781864224577058-825-0/krb5kdc/kudu_127.0.206.65.keytab
--principal=kudu/127.0.206.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.206.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781864224577058-825-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.206.65
--webserver_port=0
--tserver_master_addrs=127.0.206.126:42821
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19974
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.206.65
--log_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781864224577058-825-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 09:32:44 UTC on ccycloud.quasar-gbtldp.root.comops.site
build id 4455440
I20260619 10:20:56.700345 19974 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 10:20:56.700572 19974 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 10:20:56.705288 19982 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 10:20:56.705284 19979 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 10:20:56.705369 19980 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 10:20:56.705554 19974 server_base.cc:1034] running on GCE node
I20260619 10:20:56.705772 19974 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 10:20:56.706027 19974 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 10:20:56.707190 19974 hybrid_clock.cc:648] HybridClock initialized: now 1781864456707167 us; error 33 us; skew 500 ppm
Jun 19 10:20:56 dist-test-slave-6pt2 krb5kdc[19867](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864456, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 10:20:56.709550 19974 init.cc:377] Logged in from keytab as kudu/127.0.206.65@KRBTEST.COM (short username kudu)
I20260619 10:20:56.709973 19974 webserver.cc:460] Webserver started at http://127.0.206.65:36543/ using document root <none> and password file <none>
I20260619 10:20:56.710175 19974 fs_manager.cc:359] Metadata directory not provided
I20260619 10:20:56.710216 19974 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 10:20:56.710371 19974 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 10:20:56.711351 19974 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781864224577058-825-0/minicluster-data/ts-0/data/instance:
uuid: "ec875360c1f149ec9b3f5bf02dfff1cd"
format_stamp: "Formatted at 2026-06-19 10:20:56 on dist-test-slave-6pt2"
server_key: "1adda4d1d8987a509d6da0e262b008f7"
server_key_iv: "20761e51e30388f22b6fe3e016eb2360"
server_key_version: "encryptionkey@0"
I20260619 10:20:56.711683 19974 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781864224577058-825-0/minicluster-data/ts-0/wal/instance:
uuid: "ec875360c1f149ec9b3f5bf02dfff1cd"
format_stamp: "Formatted at 2026-06-19 10:20:56 on dist-test-slave-6pt2"
server_key: "1adda4d1d8987a509d6da0e262b008f7"
server_key_iv: "20761e51e30388f22b6fe3e016eb2360"
server_key_version: "encryptionkey@0"
I20260619 10:20:56.712880 19974 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260619 10:20:56.713585 19989 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 10:20:56.713762 19974 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260619 10:20:56.713850 19974 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781864224577058-825-0/minicluster-data/ts-0/data,/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781864224577058-825-0/minicluster-data/ts-0/wal
uuid: "ec875360c1f149ec9b3f5bf02dfff1cd"
format_stamp: "Formatted at 2026-06-19 10:20:56 on dist-test-slave-6pt2"
server_key: "1adda4d1d8987a509d6da0e262b008f7"
server_key_iv: "20761e51e30388f22b6fe3e016eb2360"
server_key_version: "encryptionkey@0"
I20260619 10:20:56.713930 19974 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781864224577058-825-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781864224577058-825-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781864224577058-825-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 10:20:56.722015 19974 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 10:20:56.724182 19974 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 10:20:56.724334 19974 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 10:20:56.724553 19974 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 10:20:56.724870 19974 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 10:20:56.724902 19974 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 10:20:56.724959 19974 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 10:20:56.724983 19974 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 10:20:56.730578 19974 rpc_server.cc:307] RPC server started. Bound to: 127.0.206.65:38797
I20260619 10:20:56.730638 20102 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.206.65:38797 every 8 connection(s)
I20260619 10:20:56.730942 19974 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781864224577058-825-0/minicluster-data/ts-0/data/info.pb
I20260619 10:20:56.732205 825 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTU9OP6/build/release/bin/kudu as pid 19974
I20260619 10:20:56.732292 825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781864224577058-825-0/minicluster-data/ts-0/wal/instance
I20260619 10:20:56.732484 825 external_mini_cluster.cc:1428] Setting key 30f78efbf2b2507ab7478ac8489a22dd
Jun 19 10:20:56 dist-test-slave-6pt2 krb5kdc[19867](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864456, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.65@KRBTEST.COM for kudu/127.0.206.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.0.206.66@KRBTEST.COM; defaulting to no policy
I20260619 10:20:56.740597 19970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:20:56.731607 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.65:48055 (local address 127.0.206.126:42821)
0619 10:20:56.731667 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0619 10:20:56.731669 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 10:20:56.731839 (+ 170us) server_negotiation.cc:407] Connection header received
0619 10:20:56.732877 (+ 1038us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:20:56.732878 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:20:56.732891 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:20:56.732936 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:20:56.733622 (+ 686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:20:56.734118 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:20:56.735249 (+ 1131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:20:56.735453 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:20:56.735551 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:20:56.737759 (+ 2208us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 10:20:56.737760 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 10:20:56.737761 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 10:20:56.737768 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 10:20:56.739184 (+ 1416us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:20:56.739590 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:20:56.739591 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:20:56.739592 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:20:56.739649 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:20:56.739906 (+ 257us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:20:56.739907 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:20:56.739908 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:20:56.740018 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 10:20:56.740092 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0619 10:20:56.740392 (+ 300us) server_negotiation.cc:299] Negotiation successful
0619 10:20:56.740486 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260619 10:20:56.740608 20105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:20:56.731566 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:42821 (local address 127.0.206.65:48055)
0619 10:20:56.731727 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0619 10:20:56.731763 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0619 10:20:56.732764 (+ 1001us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 10:20:56.732997 (+ 233us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 10:20:56.733001 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 10:20:56.733340 (+ 339us) client_negotiation.cc:189] Negotiated authn=SASL
0619 10:20:56.733532 (+ 192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:20:56.733536 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:20:56.734224 (+ 688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:20:56.734225 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:20:56.735158 (+ 933us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:20:56.735160 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:20:56.735560 (+ 400us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:20:56.735561 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:20:56.735684 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:20:56.736149 (+ 465us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 10:20:56.736159 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0619 10:20:56.737667 (+ 1508us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 10:20:56.739276 (+ 1609us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 10:20:56.739282 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 10:20:56.739284 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 10:20:56.739499 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 10:20:56.739716 (+ 217us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 10:20:56.739717 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 10:20:56.739717 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 10:20:56.739823 (+ 106us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 10:20:56.740084 (+ 261us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 10:20:56.740090 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 10:20:56.740220 (+ 130us) client_negotiation.cc:769] Sending connection context
0619 10:20:56.740332 (+ 112us) client_negotiation.cc:240] Negotiation successful
0619 10:20:56.740464 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":101,"threads_started":1}
I20260619 10:20:56.741080 20103 heartbeater.cc:347] Connected to a master server at 127.0.206.126:42821
I20260619 10:20:56.741178 20103 heartbeater.cc:464] Registering TS with master...
I20260619 10:20:56.741350 20103 heartbeater.cc:511] Master 127.0.206.126:42821 requested a full tablet report, sending...
I20260619 10:20:56.741843 19912 ts_manager.cc:194] Registered new tserver with Master: ec875360c1f149ec9b3f5bf02dfff1cd (127.0.206.65:38797)
I20260619 10:20:56.742539 19912 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.206.65@KRBTEST.COM'} at 127.0.206.65:48055
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.206.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.206.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781864224577058-825-0/krb5kdc/kudu_127.0.206.66.keytab.
Entry for principal kudu/127.0.206.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781864224577058-825-0/krb5kdc/kudu_127.0.206.66.keytab.
I20260619 10:20:56.755553 825 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTU9OP6/build/release/bin/kudu
/tmp/dist-test-taskTU9OP6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781864224577058-825-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781864224577058-825-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781864224577058-825-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781864224577058-825-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.206.66:0
--local_ip_for_outbound_sockets=127.0.206.66
--webserver_interface=127.0.206.66
--webserver_port=0
--tserver_master_addrs=127.0.206.126:42821
--keytab_file=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781864224577058-825-0/krb5kdc/kudu_127.0.206.66.keytab
--principal=kudu/127.0.206.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.206.84:36119
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781864224577058-825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781864224577058-825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781864224577058-825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 10:20:56.832710 20108 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 10:20:56.832885 20108 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 10:20:56.832906 20108 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 10:20:56.833963 20108 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 10:20:56.833997 20108 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 10:20:56.834021 20108 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.206.66
I20260619 10:20:56.835459 20108 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.206.84:36119
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781864224577058-825-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781864224577058-825-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781864224577058-825-0/krb5kdc/kudu_127.0.206.66.keytab
--principal=kudu/127.0.206.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.206.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781864224577058-825-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.206.66
--webserver_port=0
--tserver_master_addrs=127.0.206.126:42821
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20108
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.206.66
--log_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781864224577058-825-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 09:32:44 UTC on ccycloud.quasar-gbtldp.root.comops.site
build id 4455440
I20260619 10:20:56.835618 20108 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 10:20:56.835772 20108 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 10:20:56.840397 20113 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 10:20:56.840442 20114 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 10:20:56.840459 20108 server_base.cc:1034] running on GCE node
W20260619 10:20:56.840402 20116 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 10:20:56.840755 20108 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 10:20:56.841002 20108 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 10:20:56.842159 20108 hybrid_clock.cc:648] HybridClock initialized: now 1781864456842140 us; error 33 us; skew 500 ppm
Jun 19 10:20:56 dist-test-slave-6pt2 krb5kdc[19867](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864456, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 10:20:56.844561 20108 init.cc:377] Logged in from keytab as kudu/127.0.206.66@KRBTEST.COM (short username kudu)
I20260619 10:20:56.844977 20108 webserver.cc:460] Webserver started at http://127.0.206.66:44071/ using document root <none> and password file <none>
I20260619 10:20:56.845197 20108 fs_manager.cc:359] Metadata directory not provided
I20260619 10:20:56.845263 20108 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 10:20:56.845405 20108 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 10:20:56.846366 20108 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781864224577058-825-0/minicluster-data/ts-1/data/instance:
uuid: "fdb368211d444de5a5be8a5c8360001c"
format_stamp: "Formatted at 2026-06-19 10:20:56 on dist-test-slave-6pt2"
server_key: "218a5f78622e4585e4af68e448deb547"
server_key_iv: "ae68e56b2c9290f8217711bb4ada2fcb"
server_key_version: "encryptionkey@0"
I20260619 10:20:56.846742 20108 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781864224577058-825-0/minicluster-data/ts-1/wal/instance:
uuid: "fdb368211d444de5a5be8a5c8360001c"
format_stamp: "Formatted at 2026-06-19 10:20:56 on dist-test-slave-6pt2"
server_key: "218a5f78622e4585e4af68e448deb547"
server_key_iv: "ae68e56b2c9290f8217711bb4ada2fcb"
server_key_version: "encryptionkey@0"
I20260619 10:20:56.848076 20108 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260619 10:20:56.848732 20123 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 10:20:56.848897 20108 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260619 10:20:56.848989 20108 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781864224577058-825-0/minicluster-data/ts-1/data,/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781864224577058-825-0/minicluster-data/ts-1/wal
uuid: "fdb368211d444de5a5be8a5c8360001c"
format_stamp: "Formatted at 2026-06-19 10:20:56 on dist-test-slave-6pt2"
server_key: "218a5f78622e4585e4af68e448deb547"
server_key_iv: "ae68e56b2c9290f8217711bb4ada2fcb"
server_key_version: "encryptionkey@0"
I20260619 10:20:56.849061 20108 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781864224577058-825-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781864224577058-825-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781864224577058-825-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 10:20:56.864533 20108 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 10:20:56.866294 20108 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 10:20:56.866446 20108 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 10:20:56.866750 20108 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 10:20:56.867053 20108 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 10:20:56.867105 20108 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 10:20:56.867146 20108 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 10:20:56.867172 20108 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 10:20:56.872581 20108 rpc_server.cc:307] RPC server started. Bound to: 127.0.206.66:43463
I20260619 10:20:56.872663 20236 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.206.66:43463 every 8 connection(s)
I20260619 10:20:56.872938 20108 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781864224577058-825-0/minicluster-data/ts-1/data/info.pb
Jun 19 10:20:56 dist-test-slave-6pt2 krb5kdc[19867](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864456, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.66@KRBTEST.COM for kudu/127.0.206.126@KRBTEST.COM
I20260619 10:20:56.879743 825 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTU9OP6/build/release/bin/kudu as pid 20108
I20260619 10:20:56.879822 825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781864224577058-825-0/minicluster-data/ts-1/wal/instance
I20260619 10:20:56.880013 825 external_mini_cluster.cc:1428] Setting key 0ba0755248046fafce8542ce62f49f6d
I20260619 10:20:56.881333 19970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:20:56.873643 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.66:52663 (local address 127.0.206.126:42821)
0619 10:20:56.873704 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0619 10:20:56.873706 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 10:20:56.873860 (+ 154us) server_negotiation.cc:407] Connection header received
0619 10:20:56.874779 (+ 919us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:20:56.874780 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:20:56.874789 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:20:56.874845 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:20:56.875363 (+ 518us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:20:56.875831 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:20:56.876734 (+ 903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:20:56.876908 (+ 174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:20:56.876963 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:20:56.878811 (+ 1848us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 10:20:56.878813 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 10:20:56.878813 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 10:20:56.878819 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 10:20:56.880011 (+ 1192us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:20:56.880441 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:20:56.880442 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:20:56.880442 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:20:56.880492 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:20:56.880716 (+ 224us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:20:56.880716 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:20:56.880717 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:20:56.880835 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 10:20:56.880886 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0619 10:20:56.881201 (+ 315us) server_negotiation.cc:299] Negotiation successful
0619 10:20:56.881257 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260619 10:20:56.881673 20239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:20:56.873610 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:42821 (local address 127.0.206.66:52663)
0619 10:20:56.873744 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0619 10:20:56.873781 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0619 10:20:56.874682 (+ 901us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 10:20:56.874898 (+ 216us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 10:20:56.874901 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 10:20:56.875114 (+ 213us) client_negotiation.cc:189] Negotiated authn=SASL
0619 10:20:56.875295 (+ 181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:20:56.875299 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:20:56.875903 (+ 604us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:20:56.875904 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:20:56.876663 (+ 759us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:20:56.876664 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:20:56.876982 (+ 318us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:20:56.876983 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:20:56.877073 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:20:56.877457 (+ 384us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 10:20:56.877464 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0619 10:20:56.878736 (+ 1272us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 10:20:56.880101 (+ 1365us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 10:20:56.880109 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 10:20:56.880110 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 10:20:56.880375 (+ 265us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 10:20:56.880550 (+ 175us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 10:20:56.880551 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 10:20:56.880551 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 10:20:56.880665 (+ 114us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 10:20:56.880898 (+ 233us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 10:20:56.880904 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 10:20:56.881059 (+ 155us) client_negotiation.cc:769] Sending connection context
0619 10:20:56.881183 (+ 124us) client_negotiation.cc:240] Negotiation successful
0619 10:20:56.881363 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":97,"thread_start_us":73,"threads_started":1}
I20260619 10:20:56.882637 20237 heartbeater.cc:347] Connected to a master server at 127.0.206.126:42821
I20260619 10:20:56.882720 20237 heartbeater.cc:464] Registering TS with master...
I20260619 10:20:56.882899 20237 heartbeater.cc:511] Master 127.0.206.126:42821 requested a full tablet report, sending...
I20260619 10:20:56.883242 19912 ts_manager.cc:194] Registered new tserver with Master: fdb368211d444de5a5be8a5c8360001c (127.0.206.66:43463)
I20260619 10:20:56.883635 19912 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.206.66@KRBTEST.COM'} at 127.0.206.66:52663
WARNING: no policy specified for kudu/127.0.206.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.206.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.206.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781864224577058-825-0/krb5kdc/kudu_127.0.206.67.keytab.
Entry for principal kudu/127.0.206.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781864224577058-825-0/krb5kdc/kudu_127.0.206.67.keytab.
I20260619 10:20:56.902287 825 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTU9OP6/build/release/bin/kudu
/tmp/dist-test-taskTU9OP6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781864224577058-825-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781864224577058-825-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781864224577058-825-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781864224577058-825-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.206.67:0
--local_ip_for_outbound_sockets=127.0.206.67
--webserver_interface=127.0.206.67
--webserver_port=0
--tserver_master_addrs=127.0.206.126:42821
--keytab_file=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781864224577058-825-0/krb5kdc/kudu_127.0.206.67.keytab
--principal=kudu/127.0.206.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.206.84:36119
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781864224577058-825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781864224577058-825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781864224577058-825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 10:20:56.980012 20242 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 10:20:56.980190 20242 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 10:20:56.980212 20242 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 10:20:56.981289 20242 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 10:20:56.981328 20242 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 10:20:56.981359 20242 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.206.67
I20260619 10:20:56.982784 20242 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.206.84:36119
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781864224577058-825-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781864224577058-825-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781864224577058-825-0/krb5kdc/kudu_127.0.206.67.keytab
--principal=kudu/127.0.206.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.206.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781864224577058-825-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.206.67
--webserver_port=0
--tserver_master_addrs=127.0.206.126:42821
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20242
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.206.67
--log_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781864224577058-825-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 09:32:44 UTC on ccycloud.quasar-gbtldp.root.comops.site
build id 4455440
I20260619 10:20:56.982951 20242 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 10:20:56.983167 20242 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 10:20:56.987839 20247 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 10:20:56.987874 20250 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 10:20:56.987839 20248 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 10:20:56.988052 20242 server_base.cc:1034] running on GCE node
I20260619 10:20:56.988294 20242 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 10:20:56.988517 20242 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 10:20:56.989673 20242 hybrid_clock.cc:648] HybridClock initialized: now 1781864456989662 us; error 43 us; skew 500 ppm
Jun 19 10:20:56 dist-test-slave-6pt2 krb5kdc[19867](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864456, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 10:20:56.992007 20242 init.cc:377] Logged in from keytab as kudu/127.0.206.67@KRBTEST.COM (short username kudu)
I20260619 10:20:56.992440 20242 webserver.cc:460] Webserver started at http://127.0.206.67:43917/ using document root <none> and password file <none>
I20260619 10:20:56.992643 20242 fs_manager.cc:359] Metadata directory not provided
I20260619 10:20:56.992704 20242 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 10:20:56.992844 20242 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 10:20:56.993994 20242 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781864224577058-825-0/minicluster-data/ts-2/data/instance:
uuid: "f809d6d872514aaa8eb8c9bce6c7c885"
format_stamp: "Formatted at 2026-06-19 10:20:56 on dist-test-slave-6pt2"
server_key: "bfdcdb0ae8d0fb6503b2b066e7a22d6d"
server_key_iv: "2b66a2a891c6a7d76e6a90a9fa933b27"
server_key_version: "encryptionkey@0"
I20260619 10:20:56.994422 20242 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781864224577058-825-0/minicluster-data/ts-2/wal/instance:
uuid: "f809d6d872514aaa8eb8c9bce6c7c885"
format_stamp: "Formatted at 2026-06-19 10:20:56 on dist-test-slave-6pt2"
server_key: "bfdcdb0ae8d0fb6503b2b066e7a22d6d"
server_key_iv: "2b66a2a891c6a7d76e6a90a9fa933b27"
server_key_version: "encryptionkey@0"
I20260619 10:20:56.995714 20242 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 10:20:56.996353 20257 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 10:20:56.996508 20242 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260619 10:20:56.996585 20242 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781864224577058-825-0/minicluster-data/ts-2/data,/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781864224577058-825-0/minicluster-data/ts-2/wal
uuid: "f809d6d872514aaa8eb8c9bce6c7c885"
format_stamp: "Formatted at 2026-06-19 10:20:56 on dist-test-slave-6pt2"
server_key: "bfdcdb0ae8d0fb6503b2b066e7a22d6d"
server_key_iv: "2b66a2a891c6a7d76e6a90a9fa933b27"
server_key_version: "encryptionkey@0"
I20260619 10:20:56.996636 20242 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781864224577058-825-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781864224577058-825-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781864224577058-825-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 10:20:57.011561 20242 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 10:20:57.013232 20242 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 10:20:57.013378 20242 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 10:20:57.013607 20242 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 10:20:57.013880 20242 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 10:20:57.013911 20242 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 10:20:57.013954 20242 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 10:20:57.013983 20242 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 10:20:57.019601 20242 rpc_server.cc:307] RPC server started. Bound to: 127.0.206.67:35783
I20260619 10:20:57.019670 20370 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.206.67:35783 every 8 connection(s)
I20260619 10:20:57.019953 20242 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781864224577058-825-0/minicluster-data/ts-2/data/info.pb
Jun 19 10:20:57 dist-test-slave-6pt2 krb5kdc[19867](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864456, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.67@KRBTEST.COM for kudu/127.0.206.126@KRBTEST.COM
I20260619 10:20:57.026767 825 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTU9OP6/build/release/bin/kudu as pid 20242
I20260619 10:20:57.026919 825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781864224577058-825-0/minicluster-data/ts-2/wal/instance
I20260619 10:20:57.027127 825 external_mini_cluster.cc:1428] Setting key 95f6f120c2fad14f29989a4ccd880747
I20260619 10:20:57.028828 19970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:20:57.020640 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.67:51221 (local address 127.0.206.126:42821)
0619 10:20:57.020691 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0619 10:20:57.020693 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 10:20:57.020875 (+ 182us) server_negotiation.cc:407] Connection header received
0619 10:20:57.021705 (+ 830us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:20:57.021706 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:20:57.021716 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:20:57.021766 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:20:57.022277 (+ 511us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:20:57.022862 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:20:57.023829 (+ 967us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:20:57.024003 (+ 174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:20:57.024050 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:20:57.026038 (+ 1988us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 10:20:57.026039 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 10:20:57.026040 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 10:20:57.026046 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 10:20:57.027705 (+ 1659us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:20:57.028149 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:20:57.028150 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:20:57.028150 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:20:57.028201 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:20:57.028375 (+ 174us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:20:57.028375 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:20:57.028375 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:20:57.028471 (+ 96us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 10:20:57.028518 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0619 10:20:57.028703 (+ 185us) server_negotiation.cc:299] Negotiation successful
0619 10:20:57.028766 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260619 10:20:57.028977 20373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:20:57.020622 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:42821 (local address 127.0.206.67:51221)
0619 10:20:57.020768 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0619 10:20:57.020798 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0619 10:20:57.021624 (+ 826us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 10:20:57.021813 (+ 189us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 10:20:57.021816 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 10:20:57.022046 (+ 230us) client_negotiation.cc:189] Negotiated authn=SASL
0619 10:20:57.022213 (+ 167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:20:57.022217 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:20:57.022945 (+ 728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:20:57.022946 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:20:57.023747 (+ 801us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:20:57.023749 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:20:57.024072 (+ 323us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:20:57.024074 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:20:57.024169 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:20:57.024658 (+ 489us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 10:20:57.024666 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 10:20:57.025952 (+ 1286us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 10:20:57.027800 (+ 1848us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 10:20:57.027805 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 10:20:57.027807 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 10:20:57.028090 (+ 283us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 10:20:57.028244 (+ 154us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 10:20:57.028245 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 10:20:57.028245 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 10:20:57.028326 (+ 81us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 10:20:57.028509 (+ 183us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 10:20:57.028511 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 10:20:57.028605 (+ 94us) client_negotiation.cc:769] Sending connection context
0619 10:20:57.028676 (+ 71us) client_negotiation.cc:240] Negotiation successful
0619 10:20:57.028779 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":76,"threads_started":1}
I20260619 10:20:57.029493 20371 heartbeater.cc:347] Connected to a master server at 127.0.206.126:42821
I20260619 10:20:57.029592 20371 heartbeater.cc:464] Registering TS with master...
I20260619 10:20:57.029750 20371 heartbeater.cc:511] Master 127.0.206.126:42821 requested a full tablet report, sending...
I20260619 10:20:57.030090 19912 ts_manager.cc:194] Registered new tserver with Master: f809d6d872514aaa8eb8c9bce6c7c885 (127.0.206.67:35783)
I20260619 10:20:57.030442 19912 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.0.206.67@KRBTEST.COM'} at 127.0.206.67:51221
I20260619 10:20:57.038971 825 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 10:20:57.045662 19970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:20:57.040336 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54612 (local address 127.0.206.126:42821)
0619 10:20:57.040407 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0619 10:20:57.040409 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 10:20:57.040509 (+ 100us) server_negotiation.cc:407] Connection header received
0619 10:20:57.040543 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:20:57.040544 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:20:57.040551 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:20:57.040600 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:20:57.041097 (+ 497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:20:57.041556 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:20:57.042232 (+ 676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:20:57.042414 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:20:57.042460 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:20:57.043540 (+ 1080us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 10:20:57.043541 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 10:20:57.043541 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 10:20:57.043547 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 10:20:57.044667 (+ 1120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:20:57.044995 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:20:57.044996 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:20:57.044996 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:20:57.045041 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:20:57.045236 (+ 195us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:20:57.045236 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:20:57.045237 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:20:57.045382 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 10:20:57.045451 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0619 10:20:57.045529 (+ 78us) server_negotiation.cc:299] Negotiation successful
0619 10:20:57.045579 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260619 10:20:57.047152 19912 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54612:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 10:20:57.048801 19912 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 10:20:57.055090 20383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:20:57.051022 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50404 (local address 127.0.206.65:38797)
0619 10:20:57.051168 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0619 10:20:57.051173 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 10:20:57.051182 (+ 9us) server_negotiation.cc:407] Connection header received
0619 10:20:57.051225 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:20:57.051227 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:20:57.051243 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:20:57.051331 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 10:20:57.051634 (+ 303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:20:57.052477 (+ 843us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:20:57.054113 (+ 1636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:20:57.054654 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:20:57.054759 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:20:57.054820 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0619 10:20:57.054962 (+ 142us) server_negotiation.cc:299] Negotiation successful
0619 10:20:57.055014 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":113,"thread_start_us":70,"threads_started":1}
I20260619 10:20:57.055150 20379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:20:57.050808 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.66:43463 (local address 127.0.0.1:44692)
0619 10:20:57.051396 (+ 588us) negotiation.cc:107] Waiting for socket to connect
0619 10:20:57.051424 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0619 10:20:57.051472 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 10:20:57.051728 (+ 256us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 10:20:57.051728 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 10:20:57.051730 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 10:20:57.051768 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:20:57.051770 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:20:57.052674 (+ 904us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:20:57.052675 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:20:57.054005 (+ 1330us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:20:57.054008 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:20:57.054892 (+ 884us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:20:57.054893 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:20:57.054957 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:20:57.054961 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 10:20:57.055017 (+ 56us) client_negotiation.cc:240] Negotiation successful
0619 10:20:57.055068 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":576,"mutex_wait_us":46,"thread_start_us":85,"threads_started":1}
I20260619 10:20:57.055282 20381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:20:57.050940 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44692 (local address 127.0.206.66:43463)
0619 10:20:57.051215 (+ 275us) server_negotiation.cc:206] Beginning negotiation
0619 10:20:57.051219 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 10:20:57.051473 (+ 254us) server_negotiation.cc:407] Connection header received
0619 10:20:57.051522 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:20:57.051524 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:20:57.051599 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:20:57.051677 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 10:20:57.051823 (+ 146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:20:57.052566 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:20:57.054089 (+ 1523us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:20:57.054827 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:20:57.054894 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:20:57.054949 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0619 10:20:57.055034 (+ 85us) server_negotiation.cc:299] Negotiation successful
0619 10:20:57.055075 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":239,"thread_start_us":92,"threads_started":1}
I20260619 10:20:57.055487 20380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:20:57.050808 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.65:38797 (local address 127.0.0.1:50404)
0619 10:20:57.050951 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0619 10:20:57.050994 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0619 10:20:57.051058 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 10:20:57.051356 (+ 298us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 10:20:57.051359 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 10:20:57.051361 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 10:20:57.051438 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:20:57.051439 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:20:57.052574 (+ 1135us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:20:57.052575 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:20:57.054005 (+ 1430us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:20:57.054008 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:20:57.054762 (+ 754us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:20:57.054763 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:20:57.054839 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:20:57.054850 (+ 11us) client_negotiation.cc:769] Sending connection context
0619 10:20:57.055414 (+ 564us) client_negotiation.cc:240] Negotiation successful
0619 10:20:57.055447 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"mutex_wait_us":50,"thread_start_us":83,"threads_started":1}
I20260619 10:20:57.055932 20171 tablet_service.cc:1467] Processing CreateTablet for tablet 7e0dc7d2a427457999bf3db7dd4fc2d2 (DEFAULT_TABLE table=test-table [id=ff2cb08aee2045428f112297ca4d4f30]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 10:20:57.056174 20382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:20:57.051024 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.67:35783 (local address 127.0.0.1:48020)
0619 10:20:57.052606 (+ 1582us) negotiation.cc:107] Waiting for socket to connect
0619 10:20:57.052617 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 10:20:57.052669 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 10:20:57.052823 (+ 154us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 10:20:57.052824 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 10:20:57.052826 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 10:20:57.052908 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:20:57.052910 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:20:57.053876 (+ 966us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:20:57.053878 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:20:57.054971 (+ 1093us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:20:57.054974 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:20:57.055964 (+ 990us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:20:57.055965 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:20:57.056023 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:20:57.056026 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 10:20:57.056075 (+ 49us) client_negotiation.cc:240] Negotiation successful
0619 10:20:57.056114 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1551,"mutex_wait_us":34,"thread_start_us":73,"threads_started":1}
I20260619 10:20:57.056133 20037 tablet_service.cc:1467] Processing CreateTablet for tablet 7e0dc7d2a427457999bf3db7dd4fc2d2 (DEFAULT_TABLE table=test-table [id=ff2cb08aee2045428f112297ca4d4f30]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 10:20:57.056270 20171 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7e0dc7d2a427457999bf3db7dd4fc2d2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 10:20:57.056383 20384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:20:57.051221 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48020 (local address 127.0.206.67:35783)
0619 10:20:57.051402 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0619 10:20:57.051420 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0619 10:20:57.052663 (+ 1243us) server_negotiation.cc:407] Connection header received
0619 10:20:57.052743 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:20:57.052746 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:20:57.052761 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:20:57.052828 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 10:20:57.052968 (+ 140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:20:57.053798 (+ 830us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:20:57.055054 (+ 1256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:20:57.055746 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:20:57.055880 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:20:57.055936 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0619 10:20:57.056273 (+ 337us) server_negotiation.cc:299] Negotiation successful
0619 10:20:57.056314 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":153,"thread_start_us":86,"threads_started":1}
I20260619 10:20:57.056435 20037 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7e0dc7d2a427457999bf3db7dd4fc2d2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 10:20:57.056993 20305 tablet_service.cc:1467] Processing CreateTablet for tablet 7e0dc7d2a427457999bf3db7dd4fc2d2 (DEFAULT_TABLE table=test-table [id=ff2cb08aee2045428f112297ca4d4f30]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 10:20:57.057261 20305 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7e0dc7d2a427457999bf3db7dd4fc2d2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 10:20:57.058976 20385 tablet_bootstrap.cc:492] T 7e0dc7d2a427457999bf3db7dd4fc2d2 P ec875360c1f149ec9b3f5bf02dfff1cd: Bootstrap starting.
I20260619 10:20:57.059535 20385 tablet_bootstrap.cc:654] T 7e0dc7d2a427457999bf3db7dd4fc2d2 P ec875360c1f149ec9b3f5bf02dfff1cd: Neither blocks nor log segments found. Creating new log.
I20260619 10:20:57.059494 20386 tablet_bootstrap.cc:492] T 7e0dc7d2a427457999bf3db7dd4fc2d2 P fdb368211d444de5a5be8a5c8360001c: Bootstrap starting.
I20260619 10:20:57.060040 20385 log.cc:824] T 7e0dc7d2a427457999bf3db7dd4fc2d2 P ec875360c1f149ec9b3f5bf02dfff1cd: Log is configured to *not* fsync() on all Append() calls
I20260619 10:20:57.060462 20388 tablet_bootstrap.cc:492] T 7e0dc7d2a427457999bf3db7dd4fc2d2 P f809d6d872514aaa8eb8c9bce6c7c885: Bootstrap starting.
I20260619 10:20:57.060478 20386 tablet_bootstrap.cc:654] T 7e0dc7d2a427457999bf3db7dd4fc2d2 P fdb368211d444de5a5be8a5c8360001c: Neither blocks nor log segments found. Creating new log.
I20260619 10:20:57.061054 20386 log.cc:824] T 7e0dc7d2a427457999bf3db7dd4fc2d2 P fdb368211d444de5a5be8a5c8360001c: Log is configured to *not* fsync() on all Append() calls
I20260619 10:20:57.061110 20388 tablet_bootstrap.cc:654] T 7e0dc7d2a427457999bf3db7dd4fc2d2 P f809d6d872514aaa8eb8c9bce6c7c885: Neither blocks nor log segments found. Creating new log.
I20260619 10:20:57.061621 20388 log.cc:824] T 7e0dc7d2a427457999bf3db7dd4fc2d2 P f809d6d872514aaa8eb8c9bce6c7c885: Log is configured to *not* fsync() on all Append() calls
I20260619 10:20:57.061774 20385 tablet_bootstrap.cc:492] T 7e0dc7d2a427457999bf3db7dd4fc2d2 P ec875360c1f149ec9b3f5bf02dfff1cd: No bootstrap required, opened a new log
I20260619 10:20:57.061774 20386 tablet_bootstrap.cc:492] T 7e0dc7d2a427457999bf3db7dd4fc2d2 P fdb368211d444de5a5be8a5c8360001c: No bootstrap required, opened a new log
I20260619 10:20:57.061846 20386 ts_tablet_manager.cc:1397] T 7e0dc7d2a427457999bf3db7dd4fc2d2 P fdb368211d444de5a5be8a5c8360001c: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260619 10:20:57.061846 20385 ts_tablet_manager.cc:1397] T 7e0dc7d2a427457999bf3db7dd4fc2d2 P ec875360c1f149ec9b3f5bf02dfff1cd: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260619 10:20:57.062423 20388 tablet_bootstrap.cc:492] T 7e0dc7d2a427457999bf3db7dd4fc2d2 P f809d6d872514aaa8eb8c9bce6c7c885: No bootstrap required, opened a new log
I20260619 10:20:57.062497 20388 ts_tablet_manager.cc:1397] T 7e0dc7d2a427457999bf3db7dd4fc2d2 P f809d6d872514aaa8eb8c9bce6c7c885: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260619 10:20:57.063910 20386 raft_consensus.cc:348] T 7e0dc7d2a427457999bf3db7dd4fc2d2 P fdb368211d444de5a5be8a5c8360001c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fdb368211d444de5a5be8a5c8360001c" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 43463 } } peers { permanent_uuid: "f809d6d872514aaa8eb8c9bce6c7c885" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 35783 } } peers { permanent_uuid: "ec875360c1f149ec9b3f5bf02dfff1cd" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 38797 } }
I20260619 10:20:57.063894 20385 raft_consensus.cc:348] T 7e0dc7d2a427457999bf3db7dd4fc2d2 P ec875360c1f149ec9b3f5bf02dfff1cd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fdb368211d444de5a5be8a5c8360001c" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 43463 } } peers { permanent_uuid: "f809d6d872514aaa8eb8c9bce6c7c885" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 35783 } } peers { permanent_uuid: "ec875360c1f149ec9b3f5bf02dfff1cd" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 38797 } }
I20260619 10:20:57.064072 20385 raft_consensus.cc:374] T 7e0dc7d2a427457999bf3db7dd4fc2d2 P ec875360c1f149ec9b3f5bf02dfff1cd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 10:20:57.064072 20386 raft_consensus.cc:374] T 7e0dc7d2a427457999bf3db7dd4fc2d2 P fdb368211d444de5a5be8a5c8360001c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 10:20:57.064109 20386 raft_consensus.cc:729] T 7e0dc7d2a427457999bf3db7dd4fc2d2 P fdb368211d444de5a5be8a5c8360001c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fdb368211d444de5a5be8a5c8360001c, State: Initialized, Role: FOLLOWER
I20260619 10:20:57.064109 20385 raft_consensus.cc:729] T 7e0dc7d2a427457999bf3db7dd4fc2d2 P ec875360c1f149ec9b3f5bf02dfff1cd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ec875360c1f149ec9b3f5bf02dfff1cd, State: Initialized, Role: FOLLOWER
I20260619 10:20:57.064004 20388 raft_consensus.cc:348] T 7e0dc7d2a427457999bf3db7dd4fc2d2 P f809d6d872514aaa8eb8c9bce6c7c885 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fdb368211d444de5a5be8a5c8360001c" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 43463 } } peers { permanent_uuid: "f809d6d872514aaa8eb8c9bce6c7c885" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 35783 } } peers { permanent_uuid: "ec875360c1f149ec9b3f5bf02dfff1cd" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 38797 } }
I20260619 10:20:57.064164 20388 raft_consensus.cc:374] T 7e0dc7d2a427457999bf3db7dd4fc2d2 P f809d6d872514aaa8eb8c9bce6c7c885 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 10:20:57.064198 20388 raft_consensus.cc:729] T 7e0dc7d2a427457999bf3db7dd4fc2d2 P f809d6d872514aaa8eb8c9bce6c7c885 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f809d6d872514aaa8eb8c9bce6c7c885, State: Initialized, Role: FOLLOWER
I20260619 10:20:57.064221 20385 consensus_queue.cc:260] T 7e0dc7d2a427457999bf3db7dd4fc2d2 P ec875360c1f149ec9b3f5bf02dfff1cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fdb368211d444de5a5be8a5c8360001c" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 43463 } } peers { permanent_uuid: "f809d6d872514aaa8eb8c9bce6c7c885" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 35783 } } peers { permanent_uuid: "ec875360c1f149ec9b3f5bf02dfff1cd" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 38797 } }
I20260619 10:20:57.064221 20386 consensus_queue.cc:260] T 7e0dc7d2a427457999bf3db7dd4fc2d2 P fdb368211d444de5a5be8a5c8360001c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fdb368211d444de5a5be8a5c8360001c" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 43463 } } peers { permanent_uuid: "f809d6d872514aaa8eb8c9bce6c7c885" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 35783 } } peers { permanent_uuid: "ec875360c1f149ec9b3f5bf02dfff1cd" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 38797 } }
I20260619 10:20:57.064322 20388 consensus_queue.cc:260] T 7e0dc7d2a427457999bf3db7dd4fc2d2 P f809d6d872514aaa8eb8c9bce6c7c885 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fdb368211d444de5a5be8a5c8360001c" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 43463 } } peers { permanent_uuid: "f809d6d872514aaa8eb8c9bce6c7c885" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 35783 } } peers { permanent_uuid: "ec875360c1f149ec9b3f5bf02dfff1cd" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 38797 } }
I20260619 10:20:57.064486 20386 ts_tablet_manager.cc:1428] T 7e0dc7d2a427457999bf3db7dd4fc2d2 P fdb368211d444de5a5be8a5c8360001c: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260619 10:20:57.064486 20385 ts_tablet_manager.cc:1428] T 7e0dc7d2a427457999bf3db7dd4fc2d2 P ec875360c1f149ec9b3f5bf02dfff1cd: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260619 10:20:57.064615 20237 heartbeater.cc:503] Master 127.0.206.126:42821 was elected leader, sending a full tablet report...
I20260619 10:20:57.064637 20103 heartbeater.cc:503] Master 127.0.206.126:42821 was elected leader, sending a full tablet report...
I20260619 10:20:57.064858 20388 ts_tablet_manager.cc:1428] T 7e0dc7d2a427457999bf3db7dd4fc2d2 P f809d6d872514aaa8eb8c9bce6c7c885: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260619 10:20:57.064975 20371 heartbeater.cc:503] Master 127.0.206.126:42821 was elected leader, sending a full tablet report...
W20260619 10:20:57.123991 20238 tablet.cc:2369] T 7e0dc7d2a427457999bf3db7dd4fc2d2 P fdb368211d444de5a5be8a5c8360001c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 10:20:57.168763 20393 raft_consensus.cc:482] T 7e0dc7d2a427457999bf3db7dd4fc2d2 P f809d6d872514aaa8eb8c9bce6c7c885 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 10:20:57.168879 20393 raft_consensus.cc:504] T 7e0dc7d2a427457999bf3db7dd4fc2d2 P f809d6d872514aaa8eb8c9bce6c7c885 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fdb368211d444de5a5be8a5c8360001c" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 43463 } } peers { permanent_uuid: "f809d6d872514aaa8eb8c9bce6c7c885" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 35783 } } peers { permanent_uuid: "ec875360c1f149ec9b3f5bf02dfff1cd" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 38797 } }
I20260619 10:20:57.169271 20393 leader_election.cc:288] T 7e0dc7d2a427457999bf3db7dd4fc2d2 P f809d6d872514aaa8eb8c9bce6c7c885 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fdb368211d444de5a5be8a5c8360001c (127.0.206.66:43463), ec875360c1f149ec9b3f5bf02dfff1cd (127.0.206.65:38797)
I20260619 10:20:57.173133 20383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:20:57.169538 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.67:53289 (local address 127.0.206.65:38797)
0619 10:20:57.169582 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0619 10:20:57.169585 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 10:20:57.169689 (+ 104us) server_negotiation.cc:407] Connection header received
0619 10:20:57.169717 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:20:57.169718 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:20:57.169729 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:20:57.169779 (+ 50us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 10:20:57.169906 (+ 127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:20:57.170643 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:20:57.172215 (+ 1572us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:20:57.172773 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:20:57.172853 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:20:57.172862 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0619 10:20:57.173029 (+ 167us) server_negotiation.cc:299] Negotiation successful
0619 10:20:57.173064 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260619 10:20:57.173133 20381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:20:57.169465 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.67:52959 (local address 127.0.206.66:43463)
0619 10:20:57.169512 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0619 10:20:57.169514 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 10:20:57.169522 (+ 8us) server_negotiation.cc:407] Connection header received
0619 10:20:57.169611 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:20:57.169612 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:20:57.169631 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:20:57.169711 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 10:20:57.169895 (+ 184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:20:57.170647 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:20:57.172215 (+ 1568us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:20:57.172775 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:20:57.172853 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:20:57.172862 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0619 10:20:57.173029 (+ 167us) server_negotiation.cc:299] Negotiation successful
0619 10:20:57.173064 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260619 10:20:57.173142 20373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:20:57.169409 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.66:43463 (local address 127.0.206.67:52959)
0619 10:20:57.169482 (+ 73us) negotiation.cc:107] Waiting for socket to connect
0619 10:20:57.169494 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 10:20:57.169535 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 10:20:57.169730 (+ 195us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 10:20:57.169731 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 10:20:57.169732 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 10:20:57.169804 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:20:57.169806 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:20:57.170731 (+ 925us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:20:57.170732 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:20:57.172033 (+ 1301us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:20:57.172035 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:20:57.172868 (+ 833us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:20:57.172869 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:20:57.172943 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:20:57.172944 (+ 1us) client_negotiation.cc:769] Sending connection context
0619 10:20:57.173006 (+ 62us) client_negotiation.cc:240] Negotiation successful
0619 10:20:57.173051 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":54,"mutex_wait_us":65}
I20260619 10:20:57.173156 20394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:20:57.169449 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.65:38797 (local address 127.0.206.67:53289)
0619 10:20:57.169596 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0619 10:20:57.169605 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0619 10:20:57.169656 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 10:20:57.169778 (+ 122us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 10:20:57.169778 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 10:20:57.169780 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 10:20:57.169839 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:20:57.169840 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:20:57.170731 (+ 891us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:20:57.170732 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:20:57.172033 (+ 1301us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:20:57.172035 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:20:57.172867 (+ 832us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:20:57.172868 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:20:57.172941 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:20:57.172945 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 10:20:57.173006 (+ 61us) client_negotiation.cc:240] Negotiation successful
0619 10:20:57.173051 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":132,"mutex_wait_us":82,"thread_start_us":92,"threads_started":1}
I20260619 10:20:57.173497 20191 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7e0dc7d2a427457999bf3db7dd4fc2d2" candidate_uuid: "f809d6d872514aaa8eb8c9bce6c7c885" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fdb368211d444de5a5be8a5c8360001c" is_pre_election: true
I20260619 10:20:57.173571 20057 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7e0dc7d2a427457999bf3db7dd4fc2d2" candidate_uuid: "f809d6d872514aaa8eb8c9bce6c7c885" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ec875360c1f149ec9b3f5bf02dfff1cd" is_pre_election: true
I20260619 10:20:57.173640 20191 raft_consensus.cc:2454] T 7e0dc7d2a427457999bf3db7dd4fc2d2 P fdb368211d444de5a5be8a5c8360001c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f809d6d872514aaa8eb8c9bce6c7c885 in term 0.
I20260619 10:20:57.173740 20057 raft_consensus.cc:2454] T 7e0dc7d2a427457999bf3db7dd4fc2d2 P ec875360c1f149ec9b3f5bf02dfff1cd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f809d6d872514aaa8eb8c9bce6c7c885 in term 0.
I20260619 10:20:57.173918 20260 leader_election.cc:302] T 7e0dc7d2a427457999bf3db7dd4fc2d2 P f809d6d872514aaa8eb8c9bce6c7c885 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f809d6d872514aaa8eb8c9bce6c7c885, fdb368211d444de5a5be8a5c8360001c; no voters:
I20260619 10:20:57.174062 20393 raft_consensus.cc:2781] T 7e0dc7d2a427457999bf3db7dd4fc2d2 P f809d6d872514aaa8eb8c9bce6c7c885 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 10:20:57.174116 20393 raft_consensus.cc:482] T 7e0dc7d2a427457999bf3db7dd4fc2d2 P f809d6d872514aaa8eb8c9bce6c7c885 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 10:20:57.174141 20393 raft_consensus.cc:3037] T 7e0dc7d2a427457999bf3db7dd4fc2d2 P f809d6d872514aaa8eb8c9bce6c7c885 [term 0 FOLLOWER]: Advancing to term 1
I20260619 10:20:57.174849 20393 raft_consensus.cc:504] T 7e0dc7d2a427457999bf3db7dd4fc2d2 P f809d6d872514aaa8eb8c9bce6c7c885 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fdb368211d444de5a5be8a5c8360001c" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 43463 } } peers { permanent_uuid: "f809d6d872514aaa8eb8c9bce6c7c885" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 35783 } } peers { permanent_uuid: "ec875360c1f149ec9b3f5bf02dfff1cd" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 38797 } }
I20260619 10:20:57.174978 20393 leader_election.cc:288] T 7e0dc7d2a427457999bf3db7dd4fc2d2 P f809d6d872514aaa8eb8c9bce6c7c885 [CANDIDATE]: Term 1 election: Requested vote from peers fdb368211d444de5a5be8a5c8360001c (127.0.206.66:43463), ec875360c1f149ec9b3f5bf02dfff1cd (127.0.206.65:38797)
I20260619 10:20:57.175132 20191 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7e0dc7d2a427457999bf3db7dd4fc2d2" candidate_uuid: "f809d6d872514aaa8eb8c9bce6c7c885" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fdb368211d444de5a5be8a5c8360001c"
I20260619 10:20:57.175175 20057 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7e0dc7d2a427457999bf3db7dd4fc2d2" candidate_uuid: "f809d6d872514aaa8eb8c9bce6c7c885" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ec875360c1f149ec9b3f5bf02dfff1cd"
I20260619 10:20:57.175218 20191 raft_consensus.cc:3037] T 7e0dc7d2a427457999bf3db7dd4fc2d2 P fdb368211d444de5a5be8a5c8360001c [term 0 FOLLOWER]: Advancing to term 1
I20260619 10:20:57.175259 20057 raft_consensus.cc:3037] T 7e0dc7d2a427457999bf3db7dd4fc2d2 P ec875360c1f149ec9b3f5bf02dfff1cd [term 0 FOLLOWER]: Advancing to term 1
I20260619 10:20:57.175859 20191 raft_consensus.cc:2454] T 7e0dc7d2a427457999bf3db7dd4fc2d2 P fdb368211d444de5a5be8a5c8360001c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f809d6d872514aaa8eb8c9bce6c7c885 in term 1.
I20260619 10:20:57.175976 20057 raft_consensus.cc:2454] T 7e0dc7d2a427457999bf3db7dd4fc2d2 P ec875360c1f149ec9b3f5bf02dfff1cd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f809d6d872514aaa8eb8c9bce6c7c885 in term 1.
I20260619 10:20:57.176059 20260 leader_election.cc:302] T 7e0dc7d2a427457999bf3db7dd4fc2d2 P f809d6d872514aaa8eb8c9bce6c7c885 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f809d6d872514aaa8eb8c9bce6c7c885, fdb368211d444de5a5be8a5c8360001c; no voters:
I20260619 10:20:57.176143 20393 raft_consensus.cc:2781] T 7e0dc7d2a427457999bf3db7dd4fc2d2 P f809d6d872514aaa8eb8c9bce6c7c885 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 10:20:57.176329 20393 raft_consensus.cc:686] T 7e0dc7d2a427457999bf3db7dd4fc2d2 P f809d6d872514aaa8eb8c9bce6c7c885 [term 1 LEADER]: Becoming Leader. State: Replica: f809d6d872514aaa8eb8c9bce6c7c885, State: Running, Role: LEADER
I20260619 10:20:57.176451 20393 consensus_queue.cc:237] T 7e0dc7d2a427457999bf3db7dd4fc2d2 P f809d6d872514aaa8eb8c9bce6c7c885 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fdb368211d444de5a5be8a5c8360001c" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 43463 } } peers { permanent_uuid: "f809d6d872514aaa8eb8c9bce6c7c885" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 35783 } } peers { permanent_uuid: "ec875360c1f149ec9b3f5bf02dfff1cd" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 38797 } }
I20260619 10:20:57.177304 19912 catalog_manager.cc:5495] T 7e0dc7d2a427457999bf3db7dd4fc2d2 P f809d6d872514aaa8eb8c9bce6c7c885 reported cstate change: term changed from 0 to 1, leader changed from <none> to f809d6d872514aaa8eb8c9bce6c7c885 (127.0.206.67). New cstate: current_term: 1 leader_uuid: "f809d6d872514aaa8eb8c9bce6c7c885" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fdb368211d444de5a5be8a5c8360001c" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 43463 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f809d6d872514aaa8eb8c9bce6c7c885" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 35783 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ec875360c1f149ec9b3f5bf02dfff1cd" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 38797 } health_report { overall_health: UNKNOWN } } }
I20260619 10:20:57.197999 20384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:20:57.195658 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48032 (local address 127.0.206.67:35783)
0619 10:20:57.195717 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0619 10:20:57.195719 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 10:20:57.195726 (+ 7us) server_negotiation.cc:407] Connection header received
0619 10:20:57.195752 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:20:57.195753 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:20:57.195770 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:20:57.195852 (+ 82us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 10:20:57.195995 (+ 143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:20:57.196458 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:20:57.197204 (+ 746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:20:57.197437 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:20:57.197512 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:20:57.197685 (+ 173us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 10:20:57.197783 (+ 98us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 10:20:57.197873 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0619 10:20:57.197891 (+ 18us) server_negotiation.cc:299] Negotiation successful
0619 10:20:57.197909 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260619 10:20:57.199770 20191 raft_consensus.cc:1261] T 7e0dc7d2a427457999bf3db7dd4fc2d2 P fdb368211d444de5a5be8a5c8360001c [term 1 FOLLOWER]: Refusing update from remote peer f809d6d872514aaa8eb8c9bce6c7c885: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 10:20:57.199790 20057 raft_consensus.cc:1261] T 7e0dc7d2a427457999bf3db7dd4fc2d2 P ec875360c1f149ec9b3f5bf02dfff1cd [term 1 FOLLOWER]: Refusing update from remote peer f809d6d872514aaa8eb8c9bce6c7c885: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 10:20:57.200057 20393 consensus_queue.cc:1035] T 7e0dc7d2a427457999bf3db7dd4fc2d2 P f809d6d872514aaa8eb8c9bce6c7c885 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fdb368211d444de5a5be8a5c8360001c" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 43463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 10:20:57.200212 20395 consensus_queue.cc:1035] T 7e0dc7d2a427457999bf3db7dd4fc2d2 P f809d6d872514aaa8eb8c9bce6c7c885 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec875360c1f149ec9b3f5bf02dfff1cd" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 38797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 10:20:57.202366 20398 mvcc.cc:205] Tried to move back new op lower bound from 7298516816687767552 to 7298516816595464192. Current Snapshot: MvccSnapshot[applied={T|T < 7298516816687767552}]
I20260619 10:20:57.203125 20400 mvcc.cc:205] Tried to move back new op lower bound from 7298516816687767552 to 7298516816595464192. Current Snapshot: MvccSnapshot[applied={T|T < 7298516816687767552 or (T in {7298516816687767552})}]
I20260619 10:20:57.205246 19912 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54612:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 10:20:57.205338 19912 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54612:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 10:20:57.206077 19912 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 03d2885f88484238ac53707a901ae3f7: Sending DeleteTablet for 3 replicas of tablet 7e0dc7d2a427457999bf3db7dd4fc2d2
I20260619 10:20:57.206374 20037 tablet_service.cc:1514] Processing DeleteTablet for tablet 7e0dc7d2a427457999bf3db7dd4fc2d2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 10:20:57 UTC) from {username='kudu', principal='kudu/127.0.206.126@KRBTEST.COM'} at 127.0.0.1:50404
I20260619 10:20:57.206456 20305 tablet_service.cc:1514] Processing DeleteTablet for tablet 7e0dc7d2a427457999bf3db7dd4fc2d2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 10:20:57 UTC) from {username='kudu', principal='kudu/127.0.206.126@KRBTEST.COM'} at 127.0.0.1:48020
I20260619 10:20:57.206568 20407 tablet_replica.cc:330] T 7e0dc7d2a427457999bf3db7dd4fc2d2 P ec875360c1f149ec9b3f5bf02dfff1cd: stopping tablet replica
I20260619 10:20:57.206583 20171 tablet_service.cc:1514] Processing DeleteTablet for tablet 7e0dc7d2a427457999bf3db7dd4fc2d2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 10:20:57 UTC) from {username='kudu', principal='kudu/127.0.206.126@KRBTEST.COM'} at 127.0.0.1:44692
I20260619 10:20:57.206677 20407 raft_consensus.cc:2229] T 7e0dc7d2a427457999bf3db7dd4fc2d2 P ec875360c1f149ec9b3f5bf02dfff1cd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 10:20:57.206773 20407 raft_consensus.cc:2258] T 7e0dc7d2a427457999bf3db7dd4fc2d2 P ec875360c1f149ec9b3f5bf02dfff1cd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 10:20:57.206779 825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTU9OP6/build/release/bin/kudu with pid 19974
I20260619 10:20:57.210709 20409 tablet_replica.cc:330] T 7e0dc7d2a427457999bf3db7dd4fc2d2 P fdb368211d444de5a5be8a5c8360001c: stopping tablet replica
I20260619 10:20:57.210826 20409 raft_consensus.cc:2229] T 7e0dc7d2a427457999bf3db7dd4fc2d2 P fdb368211d444de5a5be8a5c8360001c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 10:20:57.210935 20409 raft_consensus.cc:2258] T 7e0dc7d2a427457999bf3db7dd4fc2d2 P fdb368211d444de5a5be8a5c8360001c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 10:20:57.210933 20408 tablet_replica.cc:330] T 7e0dc7d2a427457999bf3db7dd4fc2d2 P f809d6d872514aaa8eb8c9bce6c7c885: stopping tablet replica
I20260619 10:20:57.211130 20408 raft_consensus.cc:2229] T 7e0dc7d2a427457999bf3db7dd4fc2d2 P f809d6d872514aaa8eb8c9bce6c7c885 [term 1 LEADER]: Raft consensus shutting down.
I20260619 10:20:57.211279 20408 raft_consensus.cc:2258] T 7e0dc7d2a427457999bf3db7dd4fc2d2 P f809d6d872514aaa8eb8c9bce6c7c885 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 10:20:57.211314 20409 ts_tablet_manager.cc:1905] T 7e0dc7d2a427457999bf3db7dd4fc2d2 P fdb368211d444de5a5be8a5c8360001c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 10:20:57.211736 20408 ts_tablet_manager.cc:1905] T 7e0dc7d2a427457999bf3db7dd4fc2d2 P f809d6d872514aaa8eb8c9bce6c7c885: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 10:20:57.212924 20409 ts_tablet_manager.cc:1918] T 7e0dc7d2a427457999bf3db7dd4fc2d2 P fdb368211d444de5a5be8a5c8360001c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 10:20:57.212996 20409 log.cc:1196] T 7e0dc7d2a427457999bf3db7dd4fc2d2 P fdb368211d444de5a5be8a5c8360001c: Deleting WAL directory at /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781864224577058-825-0/minicluster-data/ts-1/wal/wals/7e0dc7d2a427457999bf3db7dd4fc2d2
I20260619 10:20:57.213248 20409 ts_tablet_manager.cc:1939] T 7e0dc7d2a427457999bf3db7dd4fc2d2 P fdb368211d444de5a5be8a5c8360001c: Deleting consensus metadata
W20260619 10:20:57.213600 19897 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.0.206.65:38797 (error 108)
I20260619 10:20:57.213783 825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTU9OP6/build/release/bin/kudu with pid 20108
I20260619 10:20:57.214949 20382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:20:57.214071 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.65:38797 (local address 127.0.0.1:50416)
0619 10:20:57.214871 (+ 800us) negotiation.cc:107] Waiting for socket to connect
0619 10:20:57.214905 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.65:38797: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":783}
W20260619 10:20:57.215847 19897 catalog_manager.cc:4584] TS ec875360c1f149ec9b3f5bf02dfff1cd (127.0.206.65:38797): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 7e0dc7d2a427457999bf3db7dd4fc2d2: Network error: Client connection negotiation failed: client connection to 127.0.206.65:38797: connect: Connection refused (error 111)
I20260619 10:20:57.216049 20408 ts_tablet_manager.cc:1918] T 7e0dc7d2a427457999bf3db7dd4fc2d2 P f809d6d872514aaa8eb8c9bce6c7c885: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 10:20:57.216127 20408 log.cc:1196] T 7e0dc7d2a427457999bf3db7dd4fc2d2 P f809d6d872514aaa8eb8c9bce6c7c885: Deleting WAL directory at /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781864224577058-825-0/minicluster-data/ts-2/wal/wals/7e0dc7d2a427457999bf3db7dd4fc2d2
I20260619 10:20:57.216394 20408 ts_tablet_manager.cc:1939] T 7e0dc7d2a427457999bf3db7dd4fc2d2 P f809d6d872514aaa8eb8c9bce6c7c885: Deleting consensus metadata
I20260619 10:20:57.216724 19898 catalog_manager.cc:4841] TS f809d6d872514aaa8eb8c9bce6c7c885 (127.0.206.67:35783): tablet 7e0dc7d2a427457999bf3db7dd4fc2d2 (table test-table [id=ff2cb08aee2045428f112297ca4d4f30]) successfully deleted
I20260619 10:20:57.219357 20382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:20:57.219267 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.66:43463 (local address 127.0.0.1:44702)
0619 10:20:57.219313 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0619 10:20:57.219338 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.66:43463: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
I20260619 10:20:57.219362 825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTU9OP6/build/release/bin/kudu with pid 20242
I20260619 10:20:57.223881 825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTU9OP6/build/release/bin/kudu with pid 19880
2026-06-19T10:20:57Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (1895 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-06-19T10:20:57Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T10:20:57Z Disabled control of system clock
I20260619 10:20:57.249692 825 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTU9OP6/build/release/bin/kudu
/tmp/dist-test-taskTU9OP6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781864224577058-825-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781864224577058-825-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781864224577058-825-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781864224577058-825-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.206.126:45947
--webserver_interface=127.0.206.126
--webserver_port=0
--builtin_ntp_servers=127.0.206.84:44441
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.206.126:45947
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260619 10:20:57.326671 20414 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 10:20:57.326861 20414 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 10:20:57.326881 20414 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 10:20:57.327943 20414 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 10:20:57.327975 20414 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 10:20:57.327987 20414 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 10:20:57.327999 20414 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 10:20:57.328011 20414 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 10:20:57.329137 20414 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.206.84:44441
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781864224577058-825-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781864224577058-825-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.206.126:45947
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.206.126:45947
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781864224577058-825-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.206.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20414
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781864224577058-825-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 09:32:44 UTC on ccycloud.quasar-gbtldp.root.comops.site
build id 4455440
I20260619 10:20:57.329296 20414 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 10:20:57.329525 20414 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260619 10:20:57.334111 20414 server_base.cc:1034] running on GCE node
W20260619 10:20:57.334095 20419 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 10:20:57.334095 20422 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 10:20:57.334087 20420 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 10:20:57.334599 20414 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 10:20:57.334874 20414 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 10:20:57.336006 20414 hybrid_clock.cc:648] HybridClock initialized: now 1781864457335997 us; error 26 us; skew 500 ppm
I20260619 10:20:57.337118 20414 webserver.cc:460] Webserver started at http://127.0.206.126:33319/ using document root <none> and password file <none>
I20260619 10:20:57.337342 20414 fs_manager.cc:359] Metadata directory not provided
I20260619 10:20:57.337384 20414 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 10:20:57.337467 20414 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 10:20:57.338402 20414 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781864224577058-825-0/minicluster-data/master-0/data/instance:
uuid: "7896518040024518a3d48383ed7207ef"
format_stamp: "Formatted at 2026-06-19 10:20:57 on dist-test-slave-6pt2"
server_key: "7482340c9d09dff7d0e0417036f2834d"
server_key_iv: "13ecbf2000ea6dd95c87aacb0d4b4883"
server_key_version: "encryptionkey@0"
I20260619 10:20:57.338759 20414 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781864224577058-825-0/minicluster-data/master-0/wal/instance:
uuid: "7896518040024518a3d48383ed7207ef"
format_stamp: "Formatted at 2026-06-19 10:20:57 on dist-test-slave-6pt2"
server_key: "7482340c9d09dff7d0e0417036f2834d"
server_key_iv: "13ecbf2000ea6dd95c87aacb0d4b4883"
server_key_version: "encryptionkey@0"
I20260619 10:20:57.339905 20414 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260619 10:20:57.340518 20428 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 10:20:57.340716 20414 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260619 10:20:57.340796 20414 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781864224577058-825-0/minicluster-data/master-0/data,/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781864224577058-825-0/minicluster-data/master-0/wal
uuid: "7896518040024518a3d48383ed7207ef"
format_stamp: "Formatted at 2026-06-19 10:20:57 on dist-test-slave-6pt2"
server_key: "7482340c9d09dff7d0e0417036f2834d"
server_key_iv: "13ecbf2000ea6dd95c87aacb0d4b4883"
server_key_version: "encryptionkey@0"
I20260619 10:20:57.340847 20414 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781864224577058-825-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781864224577058-825-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781864224577058-825-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 10:20:57.355180 20414 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 10:20:57.355425 20414 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 10:20:57.355531 20414 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 10:20:57.359133 20414 rpc_server.cc:307] RPC server started. Bound to: 127.0.206.126:45947
I20260619 10:20:57.359190 20480 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.206.126:45947 every 8 connection(s)
I20260619 10:20:57.359499 20414 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781864224577058-825-0/minicluster-data/master-0/data/info.pb
I20260619 10:20:57.360383 20481 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 10:20:57.362938 20481 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7896518040024518a3d48383ed7207ef: Bootstrap starting.
I20260619 10:20:57.363583 20481 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7896518040024518a3d48383ed7207ef: Neither blocks nor log segments found. Creating new log.
I20260619 10:20:57.363636 825 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTU9OP6/build/release/bin/kudu as pid 20414
I20260619 10:20:57.363762 825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781864224577058-825-0/minicluster-data/master-0/wal/instance
I20260619 10:20:57.363936 20481 log.cc:824] T 00000000000000000000000000000000 P 7896518040024518a3d48383ed7207ef: Log is configured to *not* fsync() on all Append() calls
I20260619 10:20:57.363991 825 external_mini_cluster.cc:1428] Setting key 5ea81e26b723f5ddfaca6b5a1cd8a967
I20260619 10:20:57.364703 20481 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7896518040024518a3d48383ed7207ef: No bootstrap required, opened a new log
I20260619 10:20:57.366959 20481 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7896518040024518a3d48383ed7207ef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7896518040024518a3d48383ed7207ef" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 45947 } }
I20260619 10:20:57.367154 20481 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7896518040024518a3d48383ed7207ef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 10:20:57.367192 20481 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7896518040024518a3d48383ed7207ef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7896518040024518a3d48383ed7207ef, State: Initialized, Role: FOLLOWER
I20260619 10:20:57.367334 20481 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7896518040024518a3d48383ed7207ef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7896518040024518a3d48383ed7207ef" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 45947 } }
I20260619 10:20:57.367403 20481 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7896518040024518a3d48383ed7207ef [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 10:20:57.367429 20481 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7896518040024518a3d48383ed7207ef [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 10:20:57.367470 20481 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7896518040024518a3d48383ed7207ef [term 0 FOLLOWER]: Advancing to term 1
I20260619 10:20:57.368057 20481 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7896518040024518a3d48383ed7207ef [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7896518040024518a3d48383ed7207ef" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 45947 } }
I20260619 10:20:57.368177 20481 leader_election.cc:302] T 00000000000000000000000000000000 P 7896518040024518a3d48383ed7207ef [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7896518040024518a3d48383ed7207ef; no voters:
I20260619 10:20:57.368330 20481 leader_election.cc:288] T 00000000000000000000000000000000 P 7896518040024518a3d48383ed7207ef [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 10:20:57.368357 20486 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7896518040024518a3d48383ed7207ef [term 1 FOLLOWER]: Leader election won for term 1
I20260619 10:20:57.368479 20486 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7896518040024518a3d48383ed7207ef [term 1 LEADER]: Becoming Leader. State: Replica: 7896518040024518a3d48383ed7207ef, State: Running, Role: LEADER
I20260619 10:20:57.368558 20486 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7896518040024518a3d48383ed7207ef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7896518040024518a3d48383ed7207ef" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 45947 } }
I20260619 10:20:57.368580 20481 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7896518040024518a3d48383ed7207ef [sys.catalog]: configured and running, proceeding with master startup.
I20260619 10:20:57.369108 20486 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7896518040024518a3d48383ed7207ef [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7896518040024518a3d48383ed7207ef. Latest consensus state: current_term: 1 leader_uuid: "7896518040024518a3d48383ed7207ef" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7896518040024518a3d48383ed7207ef" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 45947 } } }
I20260619 10:20:57.369175 20486 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7896518040024518a3d48383ed7207ef [sys.catalog]: This master's current role is: LEADER
I20260619 10:20:57.369427 20487 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7896518040024518a3d48383ed7207ef [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7896518040024518a3d48383ed7207ef" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7896518040024518a3d48383ed7207ef" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 45947 } } }
I20260619 10:20:57.369486 20487 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7896518040024518a3d48383ed7207ef [sys.catalog]: This master's current role is: LEADER
W20260619 10:20:57.370482 20501 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 7896518040024518a3d48383ed7207ef: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 10:20:57.370754 20501 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 10:20:57.370772 20485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:20:57.364496 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40208 (local address 127.0.206.126:45947)
0619 10:20:57.364698 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0619 10:20:57.364706 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 10:20:57.364738 (+ 32us) server_negotiation.cc:407] Connection header received
0619 10:20:57.365248 (+ 510us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:20:57.365257 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:20:57.365332 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:20:57.365501 (+ 169us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:20:57.365654 (+ 153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:20:57.367630 (+ 1976us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:20:57.368489 (+ 859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:20:57.369018 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:20:57.369144 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:20:57.369862 (+ 718us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 10:20:57.369866 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 10:20:57.369866 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 10:20:57.370060 (+ 194us) server_negotiation.cc:922] calling sasl_server_start()
0619 10:20:57.370100 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 10:20:57.370114 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 10:20:57.370172 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0619 10:20:57.370464 (+ 292us) server_negotiation.cc:299] Negotiation successful
0619 10:20:57.370538 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":127,"thread_start_us":79,"threads_started":1}
I20260619 10:20:57.370824 20492 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 10:20:57.370957 20492 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 10:20:57.372663 20492 catalog_manager.cc:1347] Generated new cluster ID: 395697c67c0e4148aa44d46cb3d17921
I20260619 10:20:57.372705 20492 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 10:20:57.390308 20492 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 10:20:57.391109 20492 catalog_manager.cc:1504] Loading token signing keys...
I20260619 10:20:57.395373 20492 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7896518040024518a3d48383ed7207ef: Generated new TSK 0
I20260619 10:20:57.395560 20492 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 10:20:57.401830 825 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTU9OP6/build/release/bin/kudu
/tmp/dist-test-taskTU9OP6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781864224577058-825-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781864224577058-825-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781864224577058-825-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781864224577058-825-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.206.65:0
--local_ip_for_outbound_sockets=127.0.206.65
--webserver_interface=127.0.206.65
--webserver_port=0
--tserver_master_addrs=127.0.206.126:45947
--builtin_ntp_servers=127.0.206.84:44441
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260619 10:20:57.509948 20505 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 10:20:57.510131 20505 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 10:20:57.510152 20505 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 10:20:57.511329 20505 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 10:20:57.511366 20505 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 10:20:57.511390 20505 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.206.65
I20260619 10:20:57.512648 20505 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.206.84:44441
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781864224577058-825-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781864224577058-825-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.206.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781864224577058-825-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.206.65
--webserver_port=0
--tserver_master_addrs=127.0.206.126:45947
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20505
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.206.65
--log_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781864224577058-825-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 09:32:44 UTC on ccycloud.quasar-gbtldp.root.comops.site
build id 4455440
I20260619 10:20:57.512882 20505 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 10:20:57.513187 20505 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 10:20:57.518997 20510 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 10:20:57.519003 20511 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 10:20:57.519003 20513 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 10:20:57.519191 20505 server_base.cc:1034] running on GCE node
I20260619 10:20:57.519428 20505 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 10:20:57.519680 20505 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 10:20:57.520835 20505 hybrid_clock.cc:648] HybridClock initialized: now 1781864457520818 us; error 33 us; skew 500 ppm
I20260619 10:20:57.521974 20505 webserver.cc:460] Webserver started at http://127.0.206.65:35383/ using document root <none> and password file <none>
I20260619 10:20:57.522217 20505 fs_manager.cc:359] Metadata directory not provided
I20260619 10:20:57.522287 20505 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 10:20:57.522446 20505 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 10:20:57.523514 20505 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781864224577058-825-0/minicluster-data/ts-0/data/instance:
uuid: "fc180561687549f1b36cc99111fd826f"
format_stamp: "Formatted at 2026-06-19 10:20:57 on dist-test-slave-6pt2"
server_key: "3c145fbf42027ba61b98023001e0faae"
server_key_iv: "b73820bff8f9a10344d2a6cee38a09d9"
server_key_version: "encryptionkey@0"
I20260619 10:20:57.523859 20505 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781864224577058-825-0/minicluster-data/ts-0/wal/instance:
uuid: "fc180561687549f1b36cc99111fd826f"
format_stamp: "Formatted at 2026-06-19 10:20:57 on dist-test-slave-6pt2"
server_key: "3c145fbf42027ba61b98023001e0faae"
server_key_iv: "b73820bff8f9a10344d2a6cee38a09d9"
server_key_version: "encryptionkey@0"
I20260619 10:20:57.525151 20505 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 10:20:57.525906 20519 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 10:20:57.526124 20505 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 10:20:57.526217 20505 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781864224577058-825-0/minicluster-data/ts-0/data,/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781864224577058-825-0/minicluster-data/ts-0/wal
uuid: "fc180561687549f1b36cc99111fd826f"
format_stamp: "Formatted at 2026-06-19 10:20:57 on dist-test-slave-6pt2"
server_key: "3c145fbf42027ba61b98023001e0faae"
server_key_iv: "b73820bff8f9a10344d2a6cee38a09d9"
server_key_version: "encryptionkey@0"
I20260619 10:20:57.526299 20505 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781864224577058-825-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781864224577058-825-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781864224577058-825-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 10:20:57.538028 20505 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 10:20:57.538303 20505 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 10:20:57.538460 20505 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 10:20:57.538738 20505 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 10:20:57.539043 20505 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 10:20:57.539095 20505 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 10:20:57.539134 20505 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 10:20:57.539160 20505 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 10:20:57.544865 20505 rpc_server.cc:307] RPC server started. Bound to: 127.0.206.65:36127
I20260619 10:20:57.544965 20632 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.206.65:36127 every 8 connection(s)
I20260619 10:20:57.545264 20505 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781864224577058-825-0/minicluster-data/ts-0/data/info.pb
I20260619 10:20:57.547001 825 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTU9OP6/build/release/bin/kudu as pid 20505
I20260619 10:20:57.547109 825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781864224577058-825-0/minicluster-data/ts-0/wal/instance
I20260619 10:20:57.547276 825 external_mini_cluster.cc:1428] Setting key 163e75956828518c31b2281a2bcad084
I20260619 10:20:57.548249 825 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTU9OP6/build/release/bin/kudu
/tmp/dist-test-taskTU9OP6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781864224577058-825-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781864224577058-825-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781864224577058-825-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781864224577058-825-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.206.66:0
--local_ip_for_outbound_sockets=127.0.206.66
--webserver_interface=127.0.206.66
--webserver_port=0
--tserver_master_addrs=127.0.206.126:45947
--builtin_ntp_servers=127.0.206.84:44441
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260619 10:20:57.551263 20485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:20:57.546067 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.65:49377 (local address 127.0.206.126:45947)
0619 10:20:57.546105 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0619 10:20:57.546108 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 10:20:57.546227 (+ 119us) server_negotiation.cc:407] Connection header received
0619 10:20:57.546712 (+ 485us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:20:57.546713 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:20:57.546722 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:20:57.546764 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:20:57.547542 (+ 778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:20:57.548218 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:20:57.549706 (+ 1488us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:20:57.550037 (+ 331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:20:57.550113 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:20:57.551016 (+ 903us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 10:20:57.551017 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 10:20:57.551018 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 10:20:57.551022 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0619 10:20:57.551035 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 10:20:57.551038 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 10:20:57.551083 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0619 10:20:57.551191 (+ 108us) server_negotiation.cc:299] Negotiation successful
0619 10:20:57.551209 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260619 10:20:57.551548 20635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:20:57.545934 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:45947 (local address 127.0.206.65:49377)
0619 10:20:57.546110 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0619 10:20:57.546147 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0619 10:20:57.546617 (+ 470us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 10:20:57.546813 (+ 196us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 10:20:57.546816 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 10:20:57.546823 (+ 7us) client_negotiation.cc:189] Negotiated authn=SASL
0619 10:20:57.547365 (+ 542us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:20:57.547372 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:20:57.548312 (+ 940us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:20:57.548313 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:20:57.549621 (+ 1308us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:20:57.549624 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:20:57.550122 (+ 498us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:20:57.550123 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:20:57.550229 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:20:57.550860 (+ 631us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 10:20:57.550860 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0619 10:20:57.550897 (+ 37us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 10:20:57.550932 (+ 35us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 10:20:57.551101 (+ 169us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 10:20:57.551107 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 10:20:57.551110 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 10:20:57.551186 (+ 76us) client_negotiation.cc:240] Negotiation successful
0619 10:20:57.551292 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":87,"threads_started":1}
I20260619 10:20:57.552073 20633 heartbeater.cc:347] Connected to a master server at 127.0.206.126:45947
I20260619 10:20:57.552173 20633 heartbeater.cc:464] Registering TS with master...
I20260619 10:20:57.552368 20633 heartbeater.cc:511] Master 127.0.206.126:45947 requested a full tablet report, sending...
I20260619 10:20:57.552853 20445 ts_manager.cc:194] Registered new tserver with Master: fc180561687549f1b36cc99111fd826f (127.0.206.65:36127)
I20260619 10:20:57.553478 20445 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.206.65:49377
W20260619 10:20:57.626173 20636 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 10:20:57.626340 20636 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 10:20:57.626359 20636 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 10:20:57.627574 20636 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 10:20:57.627620 20636 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 10:20:57.627651 20636 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.206.66
I20260619 10:20:57.629065 20636 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.206.84:44441
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781864224577058-825-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781864224577058-825-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.206.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781864224577058-825-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.206.66
--webserver_port=0
--tserver_master_addrs=127.0.206.126:45947
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20636
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.206.66
--log_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781864224577058-825-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 09:32:44 UTC on ccycloud.quasar-gbtldp.root.comops.site
build id 4455440
I20260619 10:20:57.629236 20636 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 10:20:57.629452 20636 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 10:20:57.634182 20642 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 10:20:57.634184 20641 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 10:20:57.634208 20636 server_base.cc:1034] running on GCE node
W20260619 10:20:57.634182 20644 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 10:20:57.634696 20636 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 10:20:57.634953 20636 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 10:20:57.636089 20636 hybrid_clock.cc:648] HybridClock initialized: now 1781864457636079 us; error 27 us; skew 500 ppm
I20260619 10:20:57.637215 20636 webserver.cc:460] Webserver started at http://127.0.206.66:34139/ using document root <none> and password file <none>
I20260619 10:20:57.637483 20636 fs_manager.cc:359] Metadata directory not provided
I20260619 10:20:57.637527 20636 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 10:20:57.637663 20636 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 10:20:57.638760 20636 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781864224577058-825-0/minicluster-data/ts-1/data/instance:
uuid: "f165967ad0e54db08f41d5473419cf3d"
format_stamp: "Formatted at 2026-06-19 10:20:57 on dist-test-slave-6pt2"
server_key: "70ea2948d46130deea96069ce7ef8c57"
server_key_iv: "5d3d155dbc0b131fc20cb8761be8e304"
server_key_version: "encryptionkey@0"
I20260619 10:20:57.639120 20636 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781864224577058-825-0/minicluster-data/ts-1/wal/instance:
uuid: "f165967ad0e54db08f41d5473419cf3d"
format_stamp: "Formatted at 2026-06-19 10:20:57 on dist-test-slave-6pt2"
server_key: "70ea2948d46130deea96069ce7ef8c57"
server_key_iv: "5d3d155dbc0b131fc20cb8761be8e304"
server_key_version: "encryptionkey@0"
I20260619 10:20:57.640297 20636 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 10:20:57.640947 20650 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 10:20:57.641112 20636 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260619 10:20:57.641184 20636 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781864224577058-825-0/minicluster-data/ts-1/data,/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781864224577058-825-0/minicluster-data/ts-1/wal
uuid: "f165967ad0e54db08f41d5473419cf3d"
format_stamp: "Formatted at 2026-06-19 10:20:57 on dist-test-slave-6pt2"
server_key: "70ea2948d46130deea96069ce7ef8c57"
server_key_iv: "5d3d155dbc0b131fc20cb8761be8e304"
server_key_version: "encryptionkey@0"
I20260619 10:20:57.641233 20636 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781864224577058-825-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781864224577058-825-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781864224577058-825-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 10:20:57.661724 20636 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 10:20:57.661942 20636 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 10:20:57.662029 20636 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 10:20:57.662191 20636 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 10:20:57.662427 20636 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 10:20:57.662454 20636 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 10:20:57.662474 20636 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 10:20:57.662504 20636 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 10:20:57.667951 20636 rpc_server.cc:307] RPC server started. Bound to: 127.0.206.66:38115
I20260619 10:20:57.668035 20763 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.206.66:38115 every 8 connection(s)
I20260619 10:20:57.668310 20636 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781864224577058-825-0/minicluster-data/ts-1/data/info.pb
I20260619 10:20:57.673308 20485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:20:57.669009 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.66:43017 (local address 127.0.206.126:45947)
0619 10:20:57.669068 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0619 10:20:57.669070 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 10:20:57.669293 (+ 223us) server_negotiation.cc:407] Connection header received
0619 10:20:57.669816 (+ 523us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:20:57.669817 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:20:57.669830 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:20:57.669874 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:20:57.670682 (+ 808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:20:57.671183 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:20:57.672119 (+ 936us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:20:57.672329 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:20:57.672398 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:20:57.673079 (+ 681us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 10:20:57.673080 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 10:20:57.673080 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 10:20:57.673084 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0619 10:20:57.673098 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 10:20:57.673102 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 10:20:57.673143 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0619 10:20:57.673214 (+ 71us) server_negotiation.cc:299] Negotiation successful
0619 10:20:57.673237 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260619 10:20:57.673528 20766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:20:57.669005 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:45947 (local address 127.0.206.66:43017)
0619 10:20:57.669184 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0619 10:20:57.669214 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0619 10:20:57.669707 (+ 493us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 10:20:57.669947 (+ 240us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 10:20:57.669951 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 10:20:57.669962 (+ 11us) client_negotiation.cc:189] Negotiated authn=SASL
0619 10:20:57.670557 (+ 595us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:20:57.670562 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:20:57.671277 (+ 715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:20:57.671278 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:20:57.672032 (+ 754us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:20:57.672034 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:20:57.672405 (+ 371us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:20:57.672406 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:20:57.672540 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:20:57.672946 (+ 406us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 10:20:57.672947 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0619 10:20:57.672970 (+ 23us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 10:20:57.672989 (+ 19us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 10:20:57.673143 (+ 154us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 10:20:57.673147 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 10:20:57.673150 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 10:20:57.673211 (+ 61us) client_negotiation.cc:240] Negotiation successful
0619 10:20:57.673299 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":71,"threads_started":1}
I20260619 10:20:57.673617 825 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTU9OP6/build/release/bin/kudu as pid 20636
I20260619 10:20:57.673712 825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781864224577058-825-0/minicluster-data/ts-1/wal/instance
I20260619 10:20:57.673904 825 external_mini_cluster.cc:1428] Setting key 5ac00362fe4b1af4c0bc2cb6cdc5a67d
I20260619 10:20:57.674047 20764 heartbeater.cc:347] Connected to a master server at 127.0.206.126:45947
I20260619 10:20:57.674126 20764 heartbeater.cc:464] Registering TS with master...
I20260619 10:20:57.674301 20764 heartbeater.cc:511] Master 127.0.206.126:45947 requested a full tablet report, sending...
I20260619 10:20:57.674736 20445 ts_manager.cc:194] Registered new tserver with Master: f165967ad0e54db08f41d5473419cf3d (127.0.206.66:38115)
I20260619 10:20:57.674911 825 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTU9OP6/build/release/bin/kudu
/tmp/dist-test-taskTU9OP6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781864224577058-825-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781864224577058-825-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781864224577058-825-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781864224577058-825-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.206.67:0
--local_ip_for_outbound_sockets=127.0.206.67
--webserver_interface=127.0.206.67
--webserver_port=0
--tserver_master_addrs=127.0.206.126:45947
--builtin_ntp_servers=127.0.206.84:44441
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260619 10:20:57.675174 20445 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.206.66:43017
W20260619 10:20:57.752657 20767 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 10:20:57.752835 20767 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 10:20:57.752856 20767 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 10:20:57.753903 20767 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 10:20:57.753931 20767 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 10:20:57.753955 20767 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.206.67
I20260619 10:20:57.755205 20767 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.206.84:44441
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781864224577058-825-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781864224577058-825-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.206.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781864224577058-825-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.206.67
--webserver_port=0
--tserver_master_addrs=127.0.206.126:45947
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20767
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.206.67
--log_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781864224577058-825-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 09:32:44 UTC on ccycloud.quasar-gbtldp.root.comops.site
build id 4455440
I20260619 10:20:57.755375 20767 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 10:20:57.755604 20767 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 10:20:57.760273 20772 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 10:20:57.760298 20775 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 10:20:57.760298 20773 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 10:20:57.760650 20767 server_base.cc:1034] running on GCE node
I20260619 10:20:57.760830 20767 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 10:20:57.761008 20767 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 10:20:57.762171 20767 hybrid_clock.cc:648] HybridClock initialized: now 1781864457762149 us; error 37 us; skew 500 ppm
I20260619 10:20:57.763342 20767 webserver.cc:460] Webserver started at http://127.0.206.67:38693/ using document root <none> and password file <none>
I20260619 10:20:57.763582 20767 fs_manager.cc:359] Metadata directory not provided
I20260619 10:20:57.763628 20767 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 10:20:57.763777 20767 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 10:20:57.764746 20767 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781864224577058-825-0/minicluster-data/ts-2/data/instance:
uuid: "098347e695534a2d857c04765c0f6bf2"
format_stamp: "Formatted at 2026-06-19 10:20:57 on dist-test-slave-6pt2"
server_key: "6248ae9dafe49468755f19188583016e"
server_key_iv: "1a3226bc7df9fb23edd2d54f9404e0e0"
server_key_version: "encryptionkey@0"
I20260619 10:20:57.765089 20767 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781864224577058-825-0/minicluster-data/ts-2/wal/instance:
uuid: "098347e695534a2d857c04765c0f6bf2"
format_stamp: "Formatted at 2026-06-19 10:20:57 on dist-test-slave-6pt2"
server_key: "6248ae9dafe49468755f19188583016e"
server_key_iv: "1a3226bc7df9fb23edd2d54f9404e0e0"
server_key_version: "encryptionkey@0"
I20260619 10:20:57.766311 20767 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 10:20:57.767063 20781 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 10:20:57.767237 20767 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260619 10:20:57.767323 20767 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781864224577058-825-0/minicluster-data/ts-2/data,/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781864224577058-825-0/minicluster-data/ts-2/wal
uuid: "098347e695534a2d857c04765c0f6bf2"
format_stamp: "Formatted at 2026-06-19 10:20:57 on dist-test-slave-6pt2"
server_key: "6248ae9dafe49468755f19188583016e"
server_key_iv: "1a3226bc7df9fb23edd2d54f9404e0e0"
server_key_version: "encryptionkey@0"
I20260619 10:20:57.767390 20767 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781864224577058-825-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781864224577058-825-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781864224577058-825-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 10:20:57.780997 20767 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 10:20:57.781241 20767 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 10:20:57.781374 20767 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 10:20:57.781584 20767 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 10:20:57.781867 20767 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 10:20:57.781916 20767 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 10:20:57.781961 20767 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 10:20:57.781988 20767 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 10:20:57.787447 20767 rpc_server.cc:307] RPC server started. Bound to: 127.0.206.67:33783
I20260619 10:20:57.787490 20894 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.206.67:33783 every 8 connection(s)
I20260619 10:20:57.787758 20767 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781864224577058-825-0/minicluster-data/ts-2/data/info.pb
I20260619 10:20:57.789458 825 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTU9OP6/build/release/bin/kudu as pid 20767
I20260619 10:20:57.789541 825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781864224577058-825-0/minicluster-data/ts-2/wal/instance
I20260619 10:20:57.789717 825 external_mini_cluster.cc:1428] Setting key 486284b785cebe425f753332afa92b44
I20260619 10:20:57.792665 20485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:20:57.788356 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.67:33793 (local address 127.0.206.126:45947)
0619 10:20:57.788419 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0619 10:20:57.788421 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 10:20:57.788633 (+ 212us) server_negotiation.cc:407] Connection header received
0619 10:20:57.789096 (+ 463us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:20:57.789098 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:20:57.789107 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:20:57.789159 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:20:57.789905 (+ 746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:20:57.790661 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:20:57.791576 (+ 915us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:20:57.791751 (+ 175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:20:57.791800 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:20:57.792463 (+ 663us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 10:20:57.792465 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 10:20:57.792465 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 10:20:57.792469 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0619 10:20:57.792481 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 10:20:57.792485 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 10:20:57.792530 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0619 10:20:57.792592 (+ 62us) server_negotiation.cc:299] Negotiation successful
0619 10:20:57.792609 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260619 10:20:57.792873 20897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:20:57.788329 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:45947 (local address 127.0.206.67:33793)
0619 10:20:57.788505 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0619 10:20:57.788542 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0619 10:20:57.789016 (+ 474us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 10:20:57.789206 (+ 190us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 10:20:57.789209 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 10:20:57.789217 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0619 10:20:57.789793 (+ 576us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:20:57.789799 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:20:57.790755 (+ 956us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:20:57.790756 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:20:57.791509 (+ 753us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:20:57.791511 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:20:57.791822 (+ 311us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:20:57.791823 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:20:57.791920 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:20:57.792346 (+ 426us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 10:20:57.792346 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0619 10:20:57.792370 (+ 24us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 10:20:57.792390 (+ 20us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 10:20:57.792532 (+ 142us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 10:20:57.792534 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 10:20:57.792536 (+ 2us) client_negotiation.cc:769] Sending connection context
0619 10:20:57.792581 (+ 45us) client_negotiation.cc:240] Negotiation successful
0619 10:20:57.792649 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":94,"threads_started":1}
I20260619 10:20:57.793241 20895 heartbeater.cc:347] Connected to a master server at 127.0.206.126:45947
I20260619 10:20:57.793322 20895 heartbeater.cc:464] Registering TS with master...
I20260619 10:20:57.793452 20895 heartbeater.cc:511] Master 127.0.206.126:45947 requested a full tablet report, sending...
I20260619 10:20:57.793816 20445 ts_manager.cc:194] Registered new tserver with Master: 098347e695534a2d857c04765c0f6bf2 (127.0.206.67:33783)
I20260619 10:20:57.794137 20445 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.206.67:33793
I20260619 10:20:57.801554 825 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 10:20:57.805706 20485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:20:57.802962 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40224 (local address 127.0.206.126:45947)
0619 10:20:57.803018 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0619 10:20:57.803020 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 10:20:57.803112 (+ 92us) server_negotiation.cc:407] Connection header received
0619 10:20:57.803184 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:20:57.803185 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:20:57.803195 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:20:57.803249 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:20:57.803424 (+ 175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:20:57.803873 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:20:57.804624 (+ 751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:20:57.804823 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:20:57.804908 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:20:57.805474 (+ 566us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 10:20:57.805476 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 10:20:57.805476 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 10:20:57.805480 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0619 10:20:57.805492 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 10:20:57.805496 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 10:20:57.805545 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0619 10:20:57.805611 (+ 66us) server_negotiation.cc:299] Negotiation successful
0619 10:20:57.805629 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260619 10:20:57.807135 20445 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:40224:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 10:20:57.808568 20445 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 10:20:57.814153 20903 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:20:57.810398 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.67:33783 (local address 127.0.0.1:48438)
0619 10:20:57.810495 (+ 97us) negotiation.cc:107] Waiting for socket to connect
0619 10:20:57.810532 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0619 10:20:57.810628 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 10:20:57.810914 (+ 286us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 10:20:57.810917 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 10:20:57.810919 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 10:20:57.810981 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:20:57.810982 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:20:57.811933 (+ 951us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:20:57.811934 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:20:57.812933 (+ 999us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:20:57.812935 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:20:57.813881 (+ 946us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:20:57.813882 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:20:57.813954 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:20:57.813966 (+ 12us) client_negotiation.cc:769] Sending connection context
0619 10:20:57.814035 (+ 69us) client_negotiation.cc:240] Negotiation successful
0619 10:20:57.814085 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":76,"thread_start_us":57,"threads_started":1}
I20260619 10:20:57.814162 20906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:20:57.810544 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48438 (local address 127.0.206.67:33783)
0619 10:20:57.810729 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0619 10:20:57.810733 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 10:20:57.810741 (+ 8us) server_negotiation.cc:407] Connection header received
0619 10:20:57.810775 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:20:57.810777 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:20:57.810791 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:20:57.811054 (+ 263us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 10:20:57.811099 (+ 45us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:20:57.811843 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:20:57.813120 (+ 1277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:20:57.813794 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:20:57.813876 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:20:57.813957 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0619 10:20:57.814051 (+ 94us) server_negotiation.cc:299] Negotiation successful
0619 10:20:57.814085 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":158,"thread_start_us":100,"threads_started":1}
I20260619 10:20:57.814427 20904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:20:57.810431 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.66:38115 (local address 127.0.0.1:33472)
0619 10:20:57.810828 (+ 397us) negotiation.cc:107] Waiting for socket to connect
0619 10:20:57.810839 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 10:20:57.810892 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 10:20:57.811054 (+ 162us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 10:20:57.811055 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 10:20:57.811057 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 10:20:57.811165 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:20:57.811167 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:20:57.812467 (+ 1300us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:20:57.812468 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:20:57.813285 (+ 817us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:20:57.813287 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:20:57.814191 (+ 904us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:20:57.814193 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:20:57.814263 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:20:57.814268 (+ 5us) client_negotiation.cc:769] Sending connection context
0619 10:20:57.814324 (+ 56us) client_negotiation.cc:240] Negotiation successful
0619 10:20:57.814360 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":378,"thread_start_us":90,"threads_started":1}
I20260619 10:20:57.814765 20908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:20:57.810663 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33472 (local address 127.0.206.66:38115)
0619 10:20:57.810906 (+ 243us) server_negotiation.cc:206] Beginning negotiation
0619 10:20:57.810910 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 10:20:57.810918 (+ 8us) server_negotiation.cc:407] Connection header received
0619 10:20:57.810954 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:20:57.810956 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:20:57.810971 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:20:57.811050 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 10:20:57.811248 (+ 198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:20:57.812359 (+ 1111us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:20:57.813369 (+ 1010us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:20:57.814014 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:20:57.814080 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:20:57.814156 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0619 10:20:57.814588 (+ 432us) server_negotiation.cc:299] Negotiation successful
0619 10:20:57.814654 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":206,"thread_start_us":80,"threads_started":1}
I20260619 10:20:57.814885 20829 tablet_service.cc:1467] Processing CreateTablet for tablet b7f0e6fa338f416a9ed5da8ee62eb3a5 (DEFAULT_TABLE table=test-table [id=5148897ff69a4b9d94620c635181a702]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 10:20:57.815099 20905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:20:57.810436 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.65:36127 (local address 127.0.0.1:33780)
0619 10:20:57.810829 (+ 393us) negotiation.cc:107] Waiting for socket to connect
0619 10:20:57.810839 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 10:20:57.810892 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 10:20:57.811187 (+ 295us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 10:20:57.811188 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 10:20:57.811189 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 10:20:57.811222 (+ 33us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:20:57.811223 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:20:57.812469 (+ 1246us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:20:57.812470 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:20:57.813546 (+ 1076us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:20:57.813548 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:20:57.814745 (+ 1197us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:20:57.814746 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:20:57.814935 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:20:57.814939 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 10:20:57.814999 (+ 60us) client_negotiation.cc:240] Negotiation successful
0619 10:20:57.815037 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":378,"mutex_wait_us":25,"thread_start_us":79,"threads_started":1}
I20260619 10:20:57.815232 20829 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b7f0e6fa338f416a9ed5da8ee62eb3a5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 10:20:57.815366 20698 tablet_service.cc:1467] Processing CreateTablet for tablet b7f0e6fa338f416a9ed5da8ee62eb3a5 (DEFAULT_TABLE table=test-table [id=5148897ff69a4b9d94620c635181a702]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 10:20:57.815692 20698 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b7f0e6fa338f416a9ed5da8ee62eb3a5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 10:20:57.817698 20909 tablet_bootstrap.cc:492] T b7f0e6fa338f416a9ed5da8ee62eb3a5 P 098347e695534a2d857c04765c0f6bf2: Bootstrap starting.
I20260619 10:20:57.818101 20910 tablet_bootstrap.cc:492] T b7f0e6fa338f416a9ed5da8ee62eb3a5 P f165967ad0e54db08f41d5473419cf3d: Bootstrap starting.
I20260619 10:20:57.818233 20907 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:20:57.810663 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33780 (local address 127.0.206.65:36127)
0619 10:20:57.810821 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0619 10:20:57.810825 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 10:20:57.810963 (+ 138us) server_negotiation.cc:407] Connection header received
0619 10:20:57.811004 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:20:57.811008 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:20:57.811046 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:20:57.811118 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 10:20:57.811294 (+ 176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:20:57.812384 (+ 1090us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:20:57.813625 (+ 1241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:20:57.814667 (+ 1042us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:20:57.814734 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:20:57.814825 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0619 10:20:57.818107 (+ 3282us) server_negotiation.cc:299] Negotiation successful
0619 10:20:57.818152 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":117,"thread_start_us":76,"threads_started":1}
I20260619 10:20:57.818429 20909 tablet_bootstrap.cc:654] T b7f0e6fa338f416a9ed5da8ee62eb3a5 P 098347e695534a2d857c04765c0f6bf2: Neither blocks nor log segments found. Creating new log.
I20260619 10:20:57.818800 20909 log.cc:824] T b7f0e6fa338f416a9ed5da8ee62eb3a5 P 098347e695534a2d857c04765c0f6bf2: Log is configured to *not* fsync() on all Append() calls
I20260619 10:20:57.818796 20567 tablet_service.cc:1467] Processing CreateTablet for tablet b7f0e6fa338f416a9ed5da8ee62eb3a5 (DEFAULT_TABLE table=test-table [id=5148897ff69a4b9d94620c635181a702]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 10:20:57.818974 20910 tablet_bootstrap.cc:654] T b7f0e6fa338f416a9ed5da8ee62eb3a5 P f165967ad0e54db08f41d5473419cf3d: Neither blocks nor log segments found. Creating new log.
I20260619 10:20:57.819100 20567 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b7f0e6fa338f416a9ed5da8ee62eb3a5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 10:20:57.819305 20910 log.cc:824] T b7f0e6fa338f416a9ed5da8ee62eb3a5 P f165967ad0e54db08f41d5473419cf3d: Log is configured to *not* fsync() on all Append() calls
I20260619 10:20:57.819600 20909 tablet_bootstrap.cc:492] T b7f0e6fa338f416a9ed5da8ee62eb3a5 P 098347e695534a2d857c04765c0f6bf2: No bootstrap required, opened a new log
I20260619 10:20:57.819680 20909 ts_tablet_manager.cc:1397] T b7f0e6fa338f416a9ed5da8ee62eb3a5 P 098347e695534a2d857c04765c0f6bf2: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260619 10:20:57.820031 20910 tablet_bootstrap.cc:492] T b7f0e6fa338f416a9ed5da8ee62eb3a5 P f165967ad0e54db08f41d5473419cf3d: No bootstrap required, opened a new log
I20260619 10:20:57.820130 20910 ts_tablet_manager.cc:1397] T b7f0e6fa338f416a9ed5da8ee62eb3a5 P f165967ad0e54db08f41d5473419cf3d: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260619 10:20:57.821684 20913 tablet_bootstrap.cc:492] T b7f0e6fa338f416a9ed5da8ee62eb3a5 P fc180561687549f1b36cc99111fd826f: Bootstrap starting.
I20260619 10:20:57.822221 20913 tablet_bootstrap.cc:654] T b7f0e6fa338f416a9ed5da8ee62eb3a5 P fc180561687549f1b36cc99111fd826f: Neither blocks nor log segments found. Creating new log.
I20260619 10:20:57.822510 20913 log.cc:824] T b7f0e6fa338f416a9ed5da8ee62eb3a5 P fc180561687549f1b36cc99111fd826f: Log is configured to *not* fsync() on all Append() calls
I20260619 10:20:57.822394 20909 raft_consensus.cc:348] T b7f0e6fa338f416a9ed5da8ee62eb3a5 P 098347e695534a2d857c04765c0f6bf2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "098347e695534a2d857c04765c0f6bf2" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 33783 } } peers { permanent_uuid: "f165967ad0e54db08f41d5473419cf3d" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 38115 } } peers { permanent_uuid: "fc180561687549f1b36cc99111fd826f" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 36127 } }
I20260619 10:20:57.822574 20909 raft_consensus.cc:374] T b7f0e6fa338f416a9ed5da8ee62eb3a5 P 098347e695534a2d857c04765c0f6bf2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 10:20:57.822664 20909 raft_consensus.cc:729] T b7f0e6fa338f416a9ed5da8ee62eb3a5 P 098347e695534a2d857c04765c0f6bf2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 098347e695534a2d857c04765c0f6bf2, State: Initialized, Role: FOLLOWER
I20260619 10:20:57.822719 20910 raft_consensus.cc:348] T b7f0e6fa338f416a9ed5da8ee62eb3a5 P f165967ad0e54db08f41d5473419cf3d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "098347e695534a2d857c04765c0f6bf2" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 33783 } } peers { permanent_uuid: "f165967ad0e54db08f41d5473419cf3d" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 38115 } } peers { permanent_uuid: "fc180561687549f1b36cc99111fd826f" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 36127 } }
I20260619 10:20:57.822784 20909 consensus_queue.cc:260] T b7f0e6fa338f416a9ed5da8ee62eb3a5 P 098347e695534a2d857c04765c0f6bf2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "098347e695534a2d857c04765c0f6bf2" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 33783 } } peers { permanent_uuid: "f165967ad0e54db08f41d5473419cf3d" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 38115 } } peers { permanent_uuid: "fc180561687549f1b36cc99111fd826f" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 36127 } }
I20260619 10:20:57.822866 20910 raft_consensus.cc:374] T b7f0e6fa338f416a9ed5da8ee62eb3a5 P f165967ad0e54db08f41d5473419cf3d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 10:20:57.822911 20910 raft_consensus.cc:729] T b7f0e6fa338f416a9ed5da8ee62eb3a5 P f165967ad0e54db08f41d5473419cf3d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f165967ad0e54db08f41d5473419cf3d, State: Initialized, Role: FOLLOWER
I20260619 10:20:57.823040 20909 ts_tablet_manager.cc:1428] T b7f0e6fa338f416a9ed5da8ee62eb3a5 P 098347e695534a2d857c04765c0f6bf2: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20260619 10:20:57.823022 20910 consensus_queue.cc:260] T b7f0e6fa338f416a9ed5da8ee62eb3a5 P f165967ad0e54db08f41d5473419cf3d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "098347e695534a2d857c04765c0f6bf2" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 33783 } } peers { permanent_uuid: "f165967ad0e54db08f41d5473419cf3d" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 38115 } } peers { permanent_uuid: "fc180561687549f1b36cc99111fd826f" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 36127 } }
I20260619 10:20:57.823089 20895 heartbeater.cc:503] Master 127.0.206.126:45947 was elected leader, sending a full tablet report...
I20260619 10:20:57.823275 20910 ts_tablet_manager.cc:1428] T b7f0e6fa338f416a9ed5da8ee62eb3a5 P f165967ad0e54db08f41d5473419cf3d: Time spent starting tablet: real 0.003s user 0.001s sys 0.002s
I20260619 10:20:57.823299 20764 heartbeater.cc:503] Master 127.0.206.126:45947 was elected leader, sending a full tablet report...
I20260619 10:20:57.824153 20913 tablet_bootstrap.cc:492] T b7f0e6fa338f416a9ed5da8ee62eb3a5 P fc180561687549f1b36cc99111fd826f: No bootstrap required, opened a new log
I20260619 10:20:57.824260 20913 ts_tablet_manager.cc:1397] T b7f0e6fa338f416a9ed5da8ee62eb3a5 P fc180561687549f1b36cc99111fd826f: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260619 10:20:57.825949 20913 raft_consensus.cc:348] T b7f0e6fa338f416a9ed5da8ee62eb3a5 P fc180561687549f1b36cc99111fd826f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "098347e695534a2d857c04765c0f6bf2" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 33783 } } peers { permanent_uuid: "f165967ad0e54db08f41d5473419cf3d" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 38115 } } peers { permanent_uuid: "fc180561687549f1b36cc99111fd826f" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 36127 } }
I20260619 10:20:57.826084 20913 raft_consensus.cc:374] T b7f0e6fa338f416a9ed5da8ee62eb3a5 P fc180561687549f1b36cc99111fd826f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 10:20:57.826122 20913 raft_consensus.cc:729] T b7f0e6fa338f416a9ed5da8ee62eb3a5 P fc180561687549f1b36cc99111fd826f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc180561687549f1b36cc99111fd826f, State: Initialized, Role: FOLLOWER
I20260619 10:20:57.826206 20913 consensus_queue.cc:260] T b7f0e6fa338f416a9ed5da8ee62eb3a5 P fc180561687549f1b36cc99111fd826f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "098347e695534a2d857c04765c0f6bf2" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 33783 } } peers { permanent_uuid: "f165967ad0e54db08f41d5473419cf3d" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 38115 } } peers { permanent_uuid: "fc180561687549f1b36cc99111fd826f" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 36127 } }
I20260619 10:20:57.826395 20913 ts_tablet_manager.cc:1428] T b7f0e6fa338f416a9ed5da8ee62eb3a5 P fc180561687549f1b36cc99111fd826f: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260619 10:20:57.826476 20633 heartbeater.cc:503] Master 127.0.206.126:45947 was elected leader, sending a full tablet report...
I20260619 10:20:57.830286 20916 raft_consensus.cc:482] T b7f0e6fa338f416a9ed5da8ee62eb3a5 P f165967ad0e54db08f41d5473419cf3d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 10:20:57.830356 20916 raft_consensus.cc:504] T b7f0e6fa338f416a9ed5da8ee62eb3a5 P f165967ad0e54db08f41d5473419cf3d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "098347e695534a2d857c04765c0f6bf2" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 33783 } } peers { permanent_uuid: "f165967ad0e54db08f41d5473419cf3d" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 38115 } } peers { permanent_uuid: "fc180561687549f1b36cc99111fd826f" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 36127 } }
I20260619 10:20:57.830595 20916 leader_election.cc:288] T b7f0e6fa338f416a9ed5da8ee62eb3a5 P f165967ad0e54db08f41d5473419cf3d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 098347e695534a2d857c04765c0f6bf2 (127.0.206.67:33783), fc180561687549f1b36cc99111fd826f (127.0.206.65:36127)
I20260619 10:20:57.833302 20766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:20:57.830757 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.67:33783 (local address 127.0.206.66:51039)
0619 10:20:57.830805 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0619 10:20:57.830840 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0619 10:20:57.830890 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 10:20:57.831005 (+ 115us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 10:20:57.831006 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 10:20:57.831008 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 10:20:57.831068 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:20:57.831070 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:20:57.831772 (+ 702us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:20:57.831773 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:20:57.832539 (+ 766us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:20:57.832541 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:20:57.833086 (+ 545us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:20:57.833087 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:20:57.833158 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:20:57.833162 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 10:20:57.833215 (+ 53us) client_negotiation.cc:240] Negotiation successful
0619 10:20:57.833254 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32}
I20260619 10:20:57.833335 20906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:20:57.830861 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.66:51039 (local address 127.0.206.67:33783)
0619 10:20:57.830900 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0619 10:20:57.830902 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 10:20:57.830912 (+ 10us) server_negotiation.cc:407] Connection header received
0619 10:20:57.830937 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:20:57.830938 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:20:57.830946 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:20:57.830998 (+ 52us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 10:20:57.831150 (+ 152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:20:57.831701 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:20:57.832620 (+ 919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:20:57.833014 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:20:57.833064 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:20:57.833070 (+ 6us) server_negotiation.cc:1035] Waiting for connection context
0619 10:20:57.833227 (+ 157us) server_negotiation.cc:299] Negotiation successful
0619 10:20:57.833262 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260619 10:20:57.833478 20907 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:20:57.830900 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.206.66:44363 (local address 127.0.206.65:36127)
0619 10:20:57.830945 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0619 10:20:57.830947 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 10:20:57.831078 (+ 131us) server_negotiation.cc:407] Connection header received
0619 10:20:57.831154 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:20:57.831155 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:20:57.831166 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:20:57.831219 (+ 53us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 10:20:57.831388 (+ 169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:20:57.831846 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:20:57.832727 (+ 881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:20:57.833110 (+ 383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:20:57.833179 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:20:57.833186 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0619 10:20:57.833365 (+ 179us) server_negotiation.cc:299] Negotiation successful
0619 10:20:57.833405 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260619 10:20:57.833537 20918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:20:57.830766 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.65:36127 (local address 127.0.206.66:44363)
0619 10:20:57.831017 (+ 251us) negotiation.cc:107] Waiting for socket to connect
0619 10:20:57.831026 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0619 10:20:57.831100 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 10:20:57.831250 (+ 150us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 10:20:57.831251 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 10:20:57.831252 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 10:20:57.831316 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:20:57.831317 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:20:57.831920 (+ 603us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:20:57.831921 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:20:57.832652 (+ 731us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:20:57.832654 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:20:57.833185 (+ 531us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:20:57.833186 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:20:57.833280 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:20:57.833283 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 10:20:57.833339 (+ 56us) client_negotiation.cc:240] Negotiation successful
0619 10:20:57.833377 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":236,"mutex_wait_us":2,"thread_start_us":101,"threads_started":1}
I20260619 10:20:57.833642 20849 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b7f0e6fa338f416a9ed5da8ee62eb3a5" candidate_uuid: "f165967ad0e54db08f41d5473419cf3d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "098347e695534a2d857c04765c0f6bf2" is_pre_election: true
I20260619 10:20:57.833768 20849 raft_consensus.cc:2454] T b7f0e6fa338f416a9ed5da8ee62eb3a5 P 098347e695534a2d857c04765c0f6bf2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f165967ad0e54db08f41d5473419cf3d in term 0.
I20260619 10:20:57.833823 20587 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b7f0e6fa338f416a9ed5da8ee62eb3a5" candidate_uuid: "f165967ad0e54db08f41d5473419cf3d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc180561687549f1b36cc99111fd826f" is_pre_election: true
I20260619 10:20:57.833932 20587 raft_consensus.cc:2454] T b7f0e6fa338f416a9ed5da8ee62eb3a5 P fc180561687549f1b36cc99111fd826f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f165967ad0e54db08f41d5473419cf3d in term 0.
I20260619 10:20:57.833973 20651 leader_election.cc:302] T b7f0e6fa338f416a9ed5da8ee62eb3a5 P f165967ad0e54db08f41d5473419cf3d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 098347e695534a2d857c04765c0f6bf2, f165967ad0e54db08f41d5473419cf3d; no voters:
I20260619 10:20:57.834087 20916 raft_consensus.cc:2781] T b7f0e6fa338f416a9ed5da8ee62eb3a5 P f165967ad0e54db08f41d5473419cf3d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 10:20:57.834141 20916 raft_consensus.cc:482] T b7f0e6fa338f416a9ed5da8ee62eb3a5 P f165967ad0e54db08f41d5473419cf3d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 10:20:57.834174 20916 raft_consensus.cc:3037] T b7f0e6fa338f416a9ed5da8ee62eb3a5 P f165967ad0e54db08f41d5473419cf3d [term 0 FOLLOWER]: Advancing to term 1
I20260619 10:20:57.834828 20916 raft_consensus.cc:504] T b7f0e6fa338f416a9ed5da8ee62eb3a5 P f165967ad0e54db08f41d5473419cf3d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "098347e695534a2d857c04765c0f6bf2" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 33783 } } peers { permanent_uuid: "f165967ad0e54db08f41d5473419cf3d" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 38115 } } peers { permanent_uuid: "fc180561687549f1b36cc99111fd826f" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 36127 } }
I20260619 10:20:57.834957 20916 leader_election.cc:288] T b7f0e6fa338f416a9ed5da8ee62eb3a5 P f165967ad0e54db08f41d5473419cf3d [CANDIDATE]: Term 1 election: Requested vote from peers 098347e695534a2d857c04765c0f6bf2 (127.0.206.67:33783), fc180561687549f1b36cc99111fd826f (127.0.206.65:36127)
I20260619 10:20:57.835109 20849 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b7f0e6fa338f416a9ed5da8ee62eb3a5" candidate_uuid: "f165967ad0e54db08f41d5473419cf3d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "098347e695534a2d857c04765c0f6bf2"
I20260619 10:20:57.835175 20849 raft_consensus.cc:3037] T b7f0e6fa338f416a9ed5da8ee62eb3a5 P 098347e695534a2d857c04765c0f6bf2 [term 0 FOLLOWER]: Advancing to term 1
I20260619 10:20:57.835129 20587 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b7f0e6fa338f416a9ed5da8ee62eb3a5" candidate_uuid: "f165967ad0e54db08f41d5473419cf3d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc180561687549f1b36cc99111fd826f"
I20260619 10:20:57.835214 20587 raft_consensus.cc:3037] T b7f0e6fa338f416a9ed5da8ee62eb3a5 P fc180561687549f1b36cc99111fd826f [term 0 FOLLOWER]: Advancing to term 1
I20260619 10:20:57.835788 20849 raft_consensus.cc:2454] T b7f0e6fa338f416a9ed5da8ee62eb3a5 P 098347e695534a2d857c04765c0f6bf2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f165967ad0e54db08f41d5473419cf3d in term 1.
I20260619 10:20:57.835793 20587 raft_consensus.cc:2454] T b7f0e6fa338f416a9ed5da8ee62eb3a5 P fc180561687549f1b36cc99111fd826f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f165967ad0e54db08f41d5473419cf3d in term 1.
I20260619 10:20:57.835955 20651 leader_election.cc:302] T b7f0e6fa338f416a9ed5da8ee62eb3a5 P f165967ad0e54db08f41d5473419cf3d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 098347e695534a2d857c04765c0f6bf2, f165967ad0e54db08f41d5473419cf3d; no voters:
I20260619 10:20:57.836040 20916 raft_consensus.cc:2781] T b7f0e6fa338f416a9ed5da8ee62eb3a5 P f165967ad0e54db08f41d5473419cf3d [term 1 FOLLOWER]: Leader election won for term 1
I20260619 10:20:57.836196 20916 raft_consensus.cc:686] T b7f0e6fa338f416a9ed5da8ee62eb3a5 P f165967ad0e54db08f41d5473419cf3d [term 1 LEADER]: Becoming Leader. State: Replica: f165967ad0e54db08f41d5473419cf3d, State: Running, Role: LEADER
I20260619 10:20:57.836303 20916 consensus_queue.cc:237] T b7f0e6fa338f416a9ed5da8ee62eb3a5 P f165967ad0e54db08f41d5473419cf3d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "098347e695534a2d857c04765c0f6bf2" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 33783 } } peers { permanent_uuid: "f165967ad0e54db08f41d5473419cf3d" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 38115 } } peers { permanent_uuid: "fc180561687549f1b36cc99111fd826f" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 36127 } }
I20260619 10:20:57.836951 20445 catalog_manager.cc:5495] T b7f0e6fa338f416a9ed5da8ee62eb3a5 P f165967ad0e54db08f41d5473419cf3d reported cstate change: term changed from 0 to 1, leader changed from <none> to f165967ad0e54db08f41d5473419cf3d (127.0.206.66). New cstate: current_term: 1 leader_uuid: "f165967ad0e54db08f41d5473419cf3d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "098347e695534a2d857c04765c0f6bf2" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 33783 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f165967ad0e54db08f41d5473419cf3d" member_type: VOTER last_known_addr { host: "127.0.206.66" port: 38115 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fc180561687549f1b36cc99111fd826f" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 36127 } health_report { overall_health: UNKNOWN } } }
I20260619 10:20:57.849227 20908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:20:57.846897 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33482 (local address 127.0.206.66:38115)
0619 10:20:57.846941 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0619 10:20:57.846943 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 10:20:57.847007 (+ 64us) server_negotiation.cc:407] Connection header received
0619 10:20:57.847092 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:20:57.847093 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:20:57.847108 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:20:57.847185 (+ 77us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 10:20:57.847299 (+ 114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:20:57.847719 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:20:57.848469 (+ 750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:20:57.848650 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:20:57.848736 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:20:57.848906 (+ 170us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 10:20:57.848999 (+ 93us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 10:20:57.849080 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0619 10:20:57.849120 (+ 40us) server_negotiation.cc:299] Negotiation successful
0619 10:20:57.849136 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260619 10:20:57.850692 20849 raft_consensus.cc:1261] T b7f0e6fa338f416a9ed5da8ee62eb3a5 P 098347e695534a2d857c04765c0f6bf2 [term 1 FOLLOWER]: Refusing update from remote peer f165967ad0e54db08f41d5473419cf3d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 10:20:57.850687 20587 raft_consensus.cc:1261] T b7f0e6fa338f416a9ed5da8ee62eb3a5 P fc180561687549f1b36cc99111fd826f [term 1 FOLLOWER]: Refusing update from remote peer f165967ad0e54db08f41d5473419cf3d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 10:20:57.850991 20919 consensus_queue.cc:1035] T b7f0e6fa338f416a9ed5da8ee62eb3a5 P f165967ad0e54db08f41d5473419cf3d [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc180561687549f1b36cc99111fd826f" member_type: VOTER last_known_addr { host: "127.0.206.65" port: 36127 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 10:20:57.851111 20916 consensus_queue.cc:1035] T b7f0e6fa338f416a9ed5da8ee62eb3a5 P f165967ad0e54db08f41d5473419cf3d [LEADER]: Connected to new peer: Peer: permanent_uuid: "098347e695534a2d857c04765c0f6bf2" member_type: VOTER last_known_addr { host: "127.0.206.67" port: 33783 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 10:20:57.853158 20922 mvcc.cc:205] Tried to move back new op lower bound from 7298516819354259456 to 7298516819298160640. Current Snapshot: MvccSnapshot[applied={T|T < 7298516819354259456}]
I20260619 10:20:57.853820 20923 mvcc.cc:205] Tried to move back new op lower bound from 7298516819354259456 to 7298516819298160640. Current Snapshot: MvccSnapshot[applied={T|T < 7298516819354259456 or (T in {7298516819354259456})}]
I20260619 10:20:57.854494 20924 mvcc.cc:205] Tried to move back new op lower bound from 7298516819354259456 to 7298516819298160640. Current Snapshot: MvccSnapshot[applied={T|T < 7298516819354259456}]
I20260619 10:20:57.855365 20444 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:40224:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 10:20:57.855443 20444 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:40224:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 10:20:57.856112 20444 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 7896518040024518a3d48383ed7207ef: Sending DeleteTablet for 3 replicas of tablet b7f0e6fa338f416a9ed5da8ee62eb3a5
I20260619 10:20:57.856432 20698 tablet_service.cc:1514] Processing DeleteTablet for tablet b7f0e6fa338f416a9ed5da8ee62eb3a5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 10:20:57 UTC) from {username='slave'} at 127.0.0.1:33472
I20260619 10:20:57.856439 20829 tablet_service.cc:1514] Processing DeleteTablet for tablet b7f0e6fa338f416a9ed5da8ee62eb3a5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 10:20:57 UTC) from {username='slave'} at 127.0.0.1:48438
I20260619 10:20:57.856492 20567 tablet_service.cc:1514] Processing DeleteTablet for tablet b7f0e6fa338f416a9ed5da8ee62eb3a5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 10:20:57 UTC) from {username='slave'} at 127.0.0.1:33780
I20260619 10:20:57.856699 20933 tablet_replica.cc:330] T b7f0e6fa338f416a9ed5da8ee62eb3a5 P 098347e695534a2d857c04765c0f6bf2: stopping tablet replica
I20260619 10:20:57.856786 20933 raft_consensus.cc:2229] T b7f0e6fa338f416a9ed5da8ee62eb3a5 P 098347e695534a2d857c04765c0f6bf2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 10:20:57.856891 20933 raft_consensus.cc:2258] T b7f0e6fa338f416a9ed5da8ee62eb3a5 P 098347e695534a2d857c04765c0f6bf2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 10:20:57.856928 825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTU9OP6/build/release/bin/kudu with pid 20505
I20260619 10:20:57.858204 20933 ts_tablet_manager.cc:1905] T b7f0e6fa338f416a9ed5da8ee62eb3a5 P 098347e695534a2d857c04765c0f6bf2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 10:20:57.859488 20932 tablet_replica.cc:330] T b7f0e6fa338f416a9ed5da8ee62eb3a5 P f165967ad0e54db08f41d5473419cf3d: stopping tablet replica
I20260619 10:20:57.859634 20932 raft_consensus.cc:2229] T b7f0e6fa338f416a9ed5da8ee62eb3a5 P f165967ad0e54db08f41d5473419cf3d [term 1 LEADER]: Raft consensus shutting down.
I20260619 10:20:57.859774 20932 raft_consensus.cc:2258] T b7f0e6fa338f416a9ed5da8ee62eb3a5 P f165967ad0e54db08f41d5473419cf3d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 10:20:57.859851 20933 ts_tablet_manager.cc:1918] T b7f0e6fa338f416a9ed5da8ee62eb3a5 P 098347e695534a2d857c04765c0f6bf2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 10:20:57.859928 20933 log.cc:1196] T b7f0e6fa338f416a9ed5da8ee62eb3a5 P 098347e695534a2d857c04765c0f6bf2: Deleting WAL directory at /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781864224577058-825-0/minicluster-data/ts-2/wal/wals/b7f0e6fa338f416a9ed5da8ee62eb3a5
I20260619 10:20:57.860193 20933 ts_tablet_manager.cc:1939] T b7f0e6fa338f416a9ed5da8ee62eb3a5 P 098347e695534a2d857c04765c0f6bf2: Deleting consensus metadata
I20260619 10:20:57.860416 20932 ts_tablet_manager.cc:1905] T b7f0e6fa338f416a9ed5da8ee62eb3a5 P f165967ad0e54db08f41d5473419cf3d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 10:20:57.860563 20429 catalog_manager.cc:4841] TS 098347e695534a2d857c04765c0f6bf2 (127.0.206.67:33783): tablet b7f0e6fa338f416a9ed5da8ee62eb3a5 (table test-table [id=5148897ff69a4b9d94620c635181a702]) successfully deleted
I20260619 10:20:57.861707 20932 ts_tablet_manager.cc:1918] T b7f0e6fa338f416a9ed5da8ee62eb3a5 P f165967ad0e54db08f41d5473419cf3d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 10:20:57.861792 20932 log.cc:1196] T b7f0e6fa338f416a9ed5da8ee62eb3a5 P f165967ad0e54db08f41d5473419cf3d: Deleting WAL directory at /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781864224577058-825-0/minicluster-data/ts-1/wal/wals/b7f0e6fa338f416a9ed5da8ee62eb3a5
I20260619 10:20:57.862001 20932 ts_tablet_manager.cc:1939] T b7f0e6fa338f416a9ed5da8ee62eb3a5 P f165967ad0e54db08f41d5473419cf3d: Deleting consensus metadata
I20260619 10:20:57.862537 20431 catalog_manager.cc:4841] TS f165967ad0e54db08f41d5473419cf3d (127.0.206.66:38115): tablet b7f0e6fa338f416a9ed5da8ee62eb3a5 (table test-table [id=5148897ff69a4b9d94620c635181a702]) successfully deleted
W20260619 10:20:57.864339 20432 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.0.206.65:36127 (error 108)
I20260619 10:20:57.864827 20905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:20:57.864690 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.65:36127 (local address 127.0.0.1:33784)
0619 10:20:57.864756 (+ 66us) negotiation.cc:107] Waiting for socket to connect
0619 10:20:57.864791 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.65:36127: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
I20260619 10:20:57.864930 825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTU9OP6/build/release/bin/kudu with pid 20636
W20260619 10:20:57.865021 20432 catalog_manager.cc:4584] TS fc180561687549f1b36cc99111fd826f (127.0.206.65:36127): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet b7f0e6fa338f416a9ed5da8ee62eb3a5: Network error: Client connection negotiation failed: client connection to 127.0.206.65:36127: connect: Connection refused (error 111)
I20260619 10:20:57.870395 825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTU9OP6/build/release/bin/kudu with pid 20767
I20260619 10:20:57.876801 825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTU9OP6/build/release/bin/kudu with pid 20414
2026-06-19T10:20:57Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (652 ms)
[----------] 6 tests from EncryptionPolicyTest (7858 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781864224577058-825-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 10:20:57 dist-test-slave-6pt2 krb5kdc[20938](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 10:20:57 dist-test-slave-6pt2 krb5kdc[20938](info): set up 2 sockets
Jun 19 10:20:57 dist-test-slave-6pt2 krb5kdc[20938](info): commencing operation
krb5kdc: starting...
W20260619 10:20:58.910668 825 mini_kdc.cc:121] Time spent starting KDC: real 1.015s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781864224577058-825-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781864224577058-825-0/krb5kdc/test-user.keytab.
Jun 19 10:20:58 dist-test-slave-6pt2 krb5kdc[20938](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864458, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T10:20:58Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T10:20:58Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781864224577058-825-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781864224577058-825-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260619 10:20:59.019073 825 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTU9OP6/build/release/bin/kudu
/tmp/dist-test-taskTU9OP6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781864224577058-825-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781864224577058-825-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781864224577058-825-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781864224577058-825-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35927
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781864224577058-825-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:41835
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:35927
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781864224577058-825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781864224577058-825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781864224577058-825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 10:20:59.096715 20951 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 10:20:59.096948 20951 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 10:20:59.096987 20951 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 10:20:59.098134 20951 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 10:20:59.098207 20951 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 10:20:59.098232 20951 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 10:20:59.098259 20951 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 10:20:59.098291 20951 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 10:20:59.099727 20951 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:41835
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781864224577058-825-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781864224577058-825-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:35927
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781864224577058-825-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35927
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781864224577058-825-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20951
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781864224577058-825-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 09:32:44 UTC on ccycloud.quasar-gbtldp.root.comops.site
build id 4455440
I20260619 10:20:59.100003 20951 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 10:20:59.100235 20951 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 10:20:59.104961 20956 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 10:20:59.104955 20957 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 10:20:59.104980 20951 server_base.cc:1034] running on GCE node
W20260619 10:20:59.104954 20959 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 10:20:59.105310 20951 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 10:20:59.105602 20951 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 10:20:59.106758 20951 hybrid_clock.cc:648] HybridClock initialized: now 1781864459106751 us; error 36 us; skew 500 ppm
Jun 19 10:20:59 dist-test-slave-6pt2 krb5kdc[20938](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864459, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 10:20:59.109144 20951 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260619 10:20:59.109547 20951 webserver.cc:460] Webserver started at http://127.0.0.1:42189/ using document root <none> and password file <none>
I20260619 10:20:59.109758 20951 fs_manager.cc:359] Metadata directory not provided
I20260619 10:20:59.109823 20951 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 10:20:59.109970 20951 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 10:20:59.111034 20951 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781864224577058-825-0/minicluster-data/master-0/data/instance:
uuid: "edad6aa89e044261a3f3b28e9fdbebef"
format_stamp: "Formatted at 2026-06-19 10:20:59 on dist-test-slave-6pt2"
server_key: "98c12f5ca6902308c258fe7b183c35a3"
server_key_iv: "97faefb34f5c9cf9802c660f910e4cf5"
server_key_version: "encryptionkey@0"
I20260619 10:20:59.111392 20951 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781864224577058-825-0/minicluster-data/master-0/wal/instance:
uuid: "edad6aa89e044261a3f3b28e9fdbebef"
format_stamp: "Formatted at 2026-06-19 10:20:59 on dist-test-slave-6pt2"
server_key: "98c12f5ca6902308c258fe7b183c35a3"
server_key_iv: "97faefb34f5c9cf9802c660f910e4cf5"
server_key_version: "encryptionkey@0"
I20260619 10:20:59.112732 20951 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 10:20:59.113461 20966 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 10:20:59.113615 20951 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 10:20:59.113713 20951 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781864224577058-825-0/minicluster-data/master-0/data,/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781864224577058-825-0/minicluster-data/master-0/wal
uuid: "edad6aa89e044261a3f3b28e9fdbebef"
format_stamp: "Formatted at 2026-06-19 10:20:59 on dist-test-slave-6pt2"
server_key: "98c12f5ca6902308c258fe7b183c35a3"
server_key_iv: "97faefb34f5c9cf9802c660f910e4cf5"
server_key_version: "encryptionkey@0"
I20260619 10:20:59.113790 20951 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781864224577058-825-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781864224577058-825-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781864224577058-825-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 10:20:59.122226 20951 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 10:20:59.124262 20951 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 10:20:59.124416 20951 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 10:20:59.128170 20951 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:35927
I20260619 10:20:59.128216 21018 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:35927 every 8 connection(s)
I20260619 10:20:59.128545 20951 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781864224577058-825-0/minicluster-data/master-0/data/info.pb
I20260619 10:20:59.129321 21019 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 10:20:59.131476 21019 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P edad6aa89e044261a3f3b28e9fdbebef: Bootstrap starting.
I20260619 10:20:59.132035 21019 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P edad6aa89e044261a3f3b28e9fdbebef: Neither blocks nor log segments found. Creating new log.
I20260619 10:20:59.132309 21019 log.cc:824] T 00000000000000000000000000000000 P edad6aa89e044261a3f3b28e9fdbebef: Log is configured to *not* fsync() on all Append() calls
I20260619 10:20:59.132977 21019 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P edad6aa89e044261a3f3b28e9fdbebef: No bootstrap required, opened a new log
I20260619 10:20:59.133453 825 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTU9OP6/build/release/bin/kudu as pid 20951
I20260619 10:20:59.133579 825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781864224577058-825-0/minicluster-data/master-0/wal/instance
I20260619 10:20:59.133772 825 external_mini_cluster.cc:1428] Setting key b2eb05768cba0922e872d45132161f89
I20260619 10:20:59.135128 21019 raft_consensus.cc:348] T 00000000000000000000000000000000 P edad6aa89e044261a3f3b28e9fdbebef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "edad6aa89e044261a3f3b28e9fdbebef" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35927 } }
I20260619 10:20:59.135280 21019 raft_consensus.cc:374] T 00000000000000000000000000000000 P edad6aa89e044261a3f3b28e9fdbebef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 10:20:59.135329 21019 raft_consensus.cc:729] T 00000000000000000000000000000000 P edad6aa89e044261a3f3b28e9fdbebef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: edad6aa89e044261a3f3b28e9fdbebef, State: Initialized, Role: FOLLOWER
I20260619 10:20:59.135452 21019 consensus_queue.cc:260] T 00000000000000000000000000000000 P edad6aa89e044261a3f3b28e9fdbebef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "edad6aa89e044261a3f3b28e9fdbebef" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35927 } }
I20260619 10:20:59.135586 21019 raft_consensus.cc:388] T 00000000000000000000000000000000 P edad6aa89e044261a3f3b28e9fdbebef [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 10:20:59.135630 21019 raft_consensus.cc:482] T 00000000000000000000000000000000 P edad6aa89e044261a3f3b28e9fdbebef [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 10:20:59.135677 21019 raft_consensus.cc:3037] T 00000000000000000000000000000000 P edad6aa89e044261a3f3b28e9fdbebef [term 0 FOLLOWER]: Advancing to term 1
I20260619 10:20:59.136173 21019 raft_consensus.cc:504] T 00000000000000000000000000000000 P edad6aa89e044261a3f3b28e9fdbebef [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "edad6aa89e044261a3f3b28e9fdbebef" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35927 } }
I20260619 10:20:59.136283 21019 leader_election.cc:302] T 00000000000000000000000000000000 P edad6aa89e044261a3f3b28e9fdbebef [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: edad6aa89e044261a3f3b28e9fdbebef; no voters:
I20260619 10:20:59.136461 21019 leader_election.cc:288] T 00000000000000000000000000000000 P edad6aa89e044261a3f3b28e9fdbebef [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 10:20:59.136529 21024 raft_consensus.cc:2781] T 00000000000000000000000000000000 P edad6aa89e044261a3f3b28e9fdbebef [term 1 FOLLOWER]: Leader election won for term 1
I20260619 10:20:59.136691 21024 raft_consensus.cc:686] T 00000000000000000000000000000000 P edad6aa89e044261a3f3b28e9fdbebef [term 1 LEADER]: Becoming Leader. State: Replica: edad6aa89e044261a3f3b28e9fdbebef, State: Running, Role: LEADER
I20260619 10:20:59.136727 21019 sys_catalog.cc:564] T 00000000000000000000000000000000 P edad6aa89e044261a3f3b28e9fdbebef [sys.catalog]: configured and running, proceeding with master startup.
I20260619 10:20:59.138135 21024 consensus_queue.cc:237] T 00000000000000000000000000000000 P edad6aa89e044261a3f3b28e9fdbebef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "edad6aa89e044261a3f3b28e9fdbebef" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35927 } }
I20260619 10:20:59.138710 21026 sys_catalog.cc:455] T 00000000000000000000000000000000 P edad6aa89e044261a3f3b28e9fdbebef [sys.catalog]: SysCatalogTable state changed. Reason: New leader edad6aa89e044261a3f3b28e9fdbebef. Latest consensus state: current_term: 1 leader_uuid: "edad6aa89e044261a3f3b28e9fdbebef" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "edad6aa89e044261a3f3b28e9fdbebef" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35927 } } }
I20260619 10:20:59.138795 21026 sys_catalog.cc:458] T 00000000000000000000000000000000 P edad6aa89e044261a3f3b28e9fdbebef [sys.catalog]: This master's current role is: LEADER
I20260619 10:20:59.139042 21039 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 10:20:59.139055 21025 sys_catalog.cc:455] T 00000000000000000000000000000000 P edad6aa89e044261a3f3b28e9fdbebef [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "edad6aa89e044261a3f3b28e9fdbebef" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "edad6aa89e044261a3f3b28e9fdbebef" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35927 } } }
I20260619 10:20:59.139175 21025 sys_catalog.cc:458] T 00000000000000000000000000000000 P edad6aa89e044261a3f3b28e9fdbebef [sys.catalog]: This master's current role is: LEADER
I20260619 10:20:59.139583 21039 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Jun 19 10:20:59 dist-test-slave-6pt2 krb5kdc[20938](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864458, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260619 10:20:59.141566 21039 catalog_manager.cc:1347] Generated new cluster ID: 93ee7b3e8be342dcb1295b62271c1f9e
I20260619 10:20:59.141619 21039 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 10:20:59.145548 21023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:20:59.134265 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58435 (local address 127.0.0.1:35927)
0619 10:20:59.134423 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0619 10:20:59.134437 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0619 10:20:59.134460 (+ 23us) server_negotiation.cc:407] Connection header received
0619 10:20:59.135263 (+ 803us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:20:59.135269 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:20:59.135335 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:20:59.135503 (+ 168us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:20:59.136505 (+ 1002us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:20:59.137590 (+ 1085us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:20:59.138425 (+ 835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:20:59.139180 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:20:59.139313 (+ 133us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:20:59.141991 (+ 2678us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 10:20:59.141995 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 10:20:59.141995 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 10:20:59.142005 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0619 10:20:59.143922 (+ 1917us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:20:59.144366 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:20:59.144369 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:20:59.144369 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:20:59.144465 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:20:59.144704 (+ 239us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:20:59.144705 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:20:59.144705 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:20:59.144988 (+ 283us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 10:20:59.145123 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0619 10:20:59.145289 (+ 166us) server_negotiation.cc:299] Negotiation successful
0619 10:20:59.145376 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":111,"thread_start_us":85,"threads_started":1}
I20260619 10:20:59.146657 825 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 10:20:59.154965 21023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:20:59.148049 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45923 (local address 127.0.0.1:35927)
0619 10:20:59.148086 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0619 10:20:59.148087 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 10:20:59.148198 (+ 111us) server_negotiation.cc:407] Connection header received
0619 10:20:59.148268 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:20:59.148269 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:20:59.148277 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:20:59.148330 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:20:59.148930 (+ 600us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:20:59.149628 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:20:59.150647 (+ 1019us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:20:59.150949 (+ 302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:20:59.151006 (+ 57us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:20:59.152299 (+ 1293us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 10:20:59.152300 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 10:20:59.152300 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 10:20:59.152307 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 10:20:59.153750 (+ 1443us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:20:59.154170 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:20:59.154171 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:20:59.154171 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:20:59.154223 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:20:59.154444 (+ 221us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:20:59.154445 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:20:59.154445 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:20:59.154553 (+ 108us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 10:20:59.154697 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0619 10:20:59.154831 (+ 134us) server_negotiation.cc:299] Negotiation successful
0619 10:20:59.154885 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260619 10:20:59.164958 21039 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 10:20:59.165601 21039 catalog_manager.cc:1504] Loading token signing keys...
I20260619 10:20:59.172036 21039 catalog_manager.cc:5868] T 00000000000000000000000000000000 P edad6aa89e044261a3f3b28e9fdbebef: Generated new TSK 0
I20260619 10:20:59.172199 21039 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 10:20:59.176460 825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTU9OP6/build/release/bin/kudu with pid 20951
W20260619 10:20:59.180704 20936 connection.cc:535] client connection to 127.0.0.1:35927 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:35927): Connection reset by peer (error 104)
2026-06-19T10:20:59Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (1298 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781864224577058-825-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 10:20:59 dist-test-slave-6pt2 krb5kdc[21050](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 10:20:59 dist-test-slave-6pt2 krb5kdc[21050](info): set up 2 sockets
Jun 19 10:20:59 dist-test-slave-6pt2 krb5kdc[21050](info): commencing operation
krb5kdc: starting...
W20260619 10:21:00.190577 825 mini_kdc.cc:121] Time spent starting KDC: real 0.996s user 0.003s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781864224577058-825-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781864224577058-825-0/krb5kdc/test-user.keytab.
Jun 19 10:21:00 dist-test-slave-6pt2 krb5kdc[21050](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864460, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T10:21:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T10:21:00Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781864224577058-825-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781864224577058-825-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260619 10:21:00.301249 825 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTU9OP6/build/release/bin/kudu
/tmp/dist-test-taskTU9OP6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781864224577058-825-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781864224577058-825-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781864224577058-825-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781864224577058-825-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36881
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781864224577058-825-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:34255
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:36881
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781864224577058-825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781864224577058-825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781864224577058-825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 10:21:00.377544 21063 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 10:21:00.377758 21063 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 10:21:00.377795 21063 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 10:21:00.378952 21063 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 10:21:00.379009 21063 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 10:21:00.379032 21063 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 10:21:00.379076 21063 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 10:21:00.379103 21063 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 10:21:00.380470 21063 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:34255
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781864224577058-825-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781864224577058-825-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:36881
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781864224577058-825-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36881
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781864224577058-825-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21063
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781864224577058-825-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 09:32:44 UTC on ccycloud.quasar-gbtldp.root.comops.site
build id 4455440
I20260619 10:21:00.380721 21063 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 10:21:00.380949 21063 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 10:21:00.385553 21069 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 10:21:00.385558 21068 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 10:21:00.385828 21063 server_base.cc:1034] running on GCE node
W20260619 10:21:00.385558 21071 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 10:21:00.386200 21063 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 10:21:00.386552 21063 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 10:21:00.387763 21063 hybrid_clock.cc:648] HybridClock initialized: now 1781864460387750 us; error 27 us; skew 500 ppm
Jun 19 10:21:00 dist-test-slave-6pt2 krb5kdc[21050](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864460, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 10:21:00.390476 21063 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260619 10:21:00.390908 21063 webserver.cc:460] Webserver started at http://127.0.0.1:35359/ using document root <none> and password file <none>
I20260619 10:21:00.391086 21063 fs_manager.cc:359] Metadata directory not provided
I20260619 10:21:00.391124 21063 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 10:21:00.391222 21063 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 10:21:00.392098 21063 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781864224577058-825-0/minicluster-data/master-0/data/instance:
uuid: "0dff2920e2484756bfa7707f9f306530"
format_stamp: "Formatted at 2026-06-19 10:21:00 on dist-test-slave-6pt2"
server_key: "fce8c8466c7a2c191580e7763e17ee29"
server_key_iv: "71c3e2b9c39dc9e2d27842bdde77044a"
server_key_version: "encryptionkey@0"
I20260619 10:21:00.392366 21063 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781864224577058-825-0/minicluster-data/master-0/wal/instance:
uuid: "0dff2920e2484756bfa7707f9f306530"
format_stamp: "Formatted at 2026-06-19 10:21:00 on dist-test-slave-6pt2"
server_key: "fce8c8466c7a2c191580e7763e17ee29"
server_key_iv: "71c3e2b9c39dc9e2d27842bdde77044a"
server_key_version: "encryptionkey@0"
I20260619 10:21:00.393558 21063 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260619 10:21:00.394218 21078 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 10:21:00.394382 21063 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260619 10:21:00.394450 21063 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781864224577058-825-0/minicluster-data/master-0/data,/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781864224577058-825-0/minicluster-data/master-0/wal
uuid: "0dff2920e2484756bfa7707f9f306530"
format_stamp: "Formatted at 2026-06-19 10:21:00 on dist-test-slave-6pt2"
server_key: "fce8c8466c7a2c191580e7763e17ee29"
server_key_iv: "71c3e2b9c39dc9e2d27842bdde77044a"
server_key_version: "encryptionkey@0"
I20260619 10:21:00.394536 21063 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781864224577058-825-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781864224577058-825-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781864224577058-825-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 10:21:00.406471 21063 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 10:21:00.408391 21063 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 10:21:00.408552 21063 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 10:21:00.412468 21063 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:36881
I20260619 10:21:00.412496 21130 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:36881 every 8 connection(s)
I20260619 10:21:00.412802 21063 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781864224577058-825-0/minicluster-data/master-0/data/info.pb
I20260619 10:21:00.413602 21131 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 10:21:00.415233 825 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTU9OP6/build/release/bin/kudu as pid 21063
I20260619 10:21:00.415349 825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781864224577058-825-0/minicluster-data/master-0/wal/instance
I20260619 10:21:00.415545 825 external_mini_cluster.cc:1428] Setting key d6c2e26c465006333faacd5c143dc403
I20260619 10:21:00.416229 21131 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0dff2920e2484756bfa7707f9f306530: Bootstrap starting.
I20260619 10:21:00.416800 21131 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0dff2920e2484756bfa7707f9f306530: Neither blocks nor log segments found. Creating new log.
I20260619 10:21:00.417129 21131 log.cc:824] T 00000000000000000000000000000000 P 0dff2920e2484756bfa7707f9f306530: Log is configured to *not* fsync() on all Append() calls
I20260619 10:21:00.417826 21131 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0dff2920e2484756bfa7707f9f306530: No bootstrap required, opened a new log
I20260619 10:21:00.420123 21131 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0dff2920e2484756bfa7707f9f306530 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0dff2920e2484756bfa7707f9f306530" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36881 } }
I20260619 10:21:00.420284 21131 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0dff2920e2484756bfa7707f9f306530 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 10:21:00.420339 21131 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0dff2920e2484756bfa7707f9f306530 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0dff2920e2484756bfa7707f9f306530, State: Initialized, Role: FOLLOWER
I20260619 10:21:00.420486 21131 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0dff2920e2484756bfa7707f9f306530 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0dff2920e2484756bfa7707f9f306530" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36881 } }
I20260619 10:21:00.420555 21131 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0dff2920e2484756bfa7707f9f306530 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 10:21:00.420576 21131 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0dff2920e2484756bfa7707f9f306530 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 10:21:00.420670 21131 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0dff2920e2484756bfa7707f9f306530 [term 0 FOLLOWER]: Advancing to term 1
I20260619 10:21:00.421012 21134 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:21:00.416018 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45551 (local address 127.0.0.1:36881)
0619 10:21:00.416185 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0619 10:21:00.416191 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 10:21:00.416214 (+ 23us) server_negotiation.cc:407] Connection header received
0619 10:21:00.416648 (+ 434us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:21:00.416654 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:21:00.416698 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:21:00.416806 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:21:00.417051 (+ 245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:21:00.418109 (+ 1058us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:21:00.419183 (+ 1074us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:21:00.419545 (+ 362us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:21:00.419621 (+ 76us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:21:00.420268 (+ 647us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 10:21:00.420271 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 10:21:00.420272 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 10:21:00.420506 (+ 234us) server_negotiation.cc:922] calling sasl_server_start()
0619 10:21:00.420537 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 10:21:00.420563 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 10:21:00.420645 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0619 10:21:00.420742 (+ 97us) server_negotiation.cc:299] Negotiation successful
0619 10:21:00.420861 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":113,"thread_start_us":81,"threads_started":1}
I20260619 10:21:00.421314 21131 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0dff2920e2484756bfa7707f9f306530 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0dff2920e2484756bfa7707f9f306530" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36881 } }
I20260619 10:21:00.421435 21131 leader_election.cc:302] T 00000000000000000000000000000000 P 0dff2920e2484756bfa7707f9f306530 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0dff2920e2484756bfa7707f9f306530; no voters:
I20260619 10:21:00.421600 21131 leader_election.cc:288] T 00000000000000000000000000000000 P 0dff2920e2484756bfa7707f9f306530 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 10:21:00.421644 21136 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0dff2920e2484756bfa7707f9f306530 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 10:21:00.421787 21136 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0dff2920e2484756bfa7707f9f306530 [term 1 LEADER]: Becoming Leader. State: Replica: 0dff2920e2484756bfa7707f9f306530, State: Running, Role: LEADER
I20260619 10:21:00.421919 825 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 10:21:00.421823 21131 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0dff2920e2484756bfa7707f9f306530 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 10:21:00.421943 21136 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0dff2920e2484756bfa7707f9f306530 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0dff2920e2484756bfa7707f9f306530" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36881 } }
I20260619 10:21:00.422675 21138 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0dff2920e2484756bfa7707f9f306530 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0dff2920e2484756bfa7707f9f306530. Latest consensus state: current_term: 1 leader_uuid: "0dff2920e2484756bfa7707f9f306530" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0dff2920e2484756bfa7707f9f306530" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36881 } } }
I20260619 10:21:00.422782 21138 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0dff2920e2484756bfa7707f9f306530 [sys.catalog]: This master's current role is: LEADER
I20260619 10:21:00.423557 21137 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0dff2920e2484756bfa7707f9f306530 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0dff2920e2484756bfa7707f9f306530" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0dff2920e2484756bfa7707f9f306530" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36881 } } }
I20260619 10:21:00.423666 21137 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0dff2920e2484756bfa7707f9f306530 [sys.catalog]: This master's current role is: LEADER
W20260619 10:21:00.424062 21157 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 0dff2920e2484756bfa7707f9f306530: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 10:21:00.424124 21157 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 10:21:00.424280 21145 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 10:21:00.424448 21145 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 10:21:00.426360 21145 catalog_manager.cc:1347] Generated new cluster ID: dbe126e2a7e54b67bad1249f51feb4cc
I20260619 10:21:00.426433 21145 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 10:21:00.427023 21134 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:21:00.423538 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38089 (local address 127.0.0.1:36881)
0619 10:21:00.423620 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0619 10:21:00.423622 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 10:21:00.423683 (+ 61us) server_negotiation.cc:407] Connection header received
0619 10:21:00.423796 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:21:00.423796 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:21:00.423811 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:21:00.423840 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:21:00.424065 (+ 225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:21:00.424774 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:21:00.425767 (+ 993us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:21:00.425963 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:21:00.426022 (+ 59us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:21:00.426737 (+ 715us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 10:21:00.426738 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 10:21:00.426738 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 10:21:00.426742 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0619 10:21:00.426755 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 10:21:00.426759 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 10:21:00.426817 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0619 10:21:00.426947 (+ 130us) server_negotiation.cc:299] Negotiation successful
0619 10:21:00.426958 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260619 10:21:00.434592 21145 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 10:21:00.435303 21145 catalog_manager.cc:1504] Loading token signing keys...
I20260619 10:21:00.442662 21145 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0dff2920e2484756bfa7707f9f306530: Generated new TSK 0
I20260619 10:21:00.442821 21145 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 10:21:00.446419 825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTU9OP6/build/release/bin/kudu with pid 21063
W20260619 10:21:00.450304 21048 connection.cc:535] client connection to 127.0.0.1:36881 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:36881): Connection reset by peer (error 104)
2026-06-19T10:21:00Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (1269 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781864224577058-825-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 10:21:00 dist-test-slave-6pt2 krb5kdc[21162](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 10:21:00 dist-test-slave-6pt2 krb5kdc[21162](info): set up 2 sockets
Jun 19 10:21:00 dist-test-slave-6pt2 krb5kdc[21162](info): commencing operation
krb5kdc: starting...
W20260619 10:21:01.454437 825 mini_kdc.cc:121] Time spent starting KDC: real 0.990s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781864224577058-825-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781864224577058-825-0/krb5kdc/test-user.keytab.
Jun 19 10:21:01 dist-test-slave-6pt2 krb5kdc[21162](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864461, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T10:21:01Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T10:21:01Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781864224577058-825-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781864224577058-825-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260619 10:21:01.565263 825 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTU9OP6/build/release/bin/kudu
/tmp/dist-test-taskTU9OP6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781864224577058-825-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781864224577058-825-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781864224577058-825-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781864224577058-825-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34381
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781864224577058-825-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:43707
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:34381
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781864224577058-825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781864224577058-825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781864224577058-825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 10:21:01.641819 21175 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 10:21:01.642030 21175 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 10:21:01.642061 21175 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 10:21:01.643236 21175 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 10:21:01.643297 21175 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 10:21:01.643321 21175 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 10:21:01.643349 21175 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 10:21:01.643379 21175 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 10:21:01.644809 21175 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:43707
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781864224577058-825-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781864224577058-825-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:34381
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781864224577058-825-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34381
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781864224577058-825-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21175
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781864224577058-825-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 09:32:44 UTC on ccycloud.quasar-gbtldp.root.comops.site
build id 4455440
I20260619 10:21:01.645078 21175 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 10:21:01.645303 21175 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260619 10:21:01.649941 21175 server_base.cc:1034] running on GCE node
W20260619 10:21:01.649879 21180 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 10:21:01.649885 21181 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 10:21:01.649881 21183 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 10:21:01.650336 21175 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 10:21:01.650663 21175 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 10:21:01.651813 21175 hybrid_clock.cc:648] HybridClock initialized: now 1781864461651801 us; error 43 us; skew 500 ppm
Jun 19 10:21:01 dist-test-slave-6pt2 krb5kdc[21162](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864461, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 10:21:01.654083 21175 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260619 10:21:01.654502 21175 webserver.cc:460] Webserver started at http://127.0.0.1:34007/ using document root <none> and password file <none>
I20260619 10:21:01.654739 21175 fs_manager.cc:359] Metadata directory not provided
I20260619 10:21:01.654783 21175 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 10:21:01.654865 21175 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 10:21:01.655762 21175 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781864224577058-825-0/minicluster-data/master-0/data/instance:
uuid: "14dd4d5a9421468cbf6a8cfcb30d09e3"
format_stamp: "Formatted at 2026-06-19 10:21:01 on dist-test-slave-6pt2"
server_key: "78e7443174ac9d6efe2884f48f64b7b8"
server_key_iv: "08d71c649bcde7befe09ee0a3ca72f8b"
server_key_version: "encryptionkey@0"
I20260619 10:21:01.656069 21175 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781864224577058-825-0/minicluster-data/master-0/wal/instance:
uuid: "14dd4d5a9421468cbf6a8cfcb30d09e3"
format_stamp: "Formatted at 2026-06-19 10:21:01 on dist-test-slave-6pt2"
server_key: "78e7443174ac9d6efe2884f48f64b7b8"
server_key_iv: "08d71c649bcde7befe09ee0a3ca72f8b"
server_key_version: "encryptionkey@0"
I20260619 10:21:01.657256 21175 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260619 10:21:01.657881 21190 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 10:21:01.658030 21175 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260619 10:21:01.658109 21175 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781864224577058-825-0/minicluster-data/master-0/data,/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781864224577058-825-0/minicluster-data/master-0/wal
uuid: "14dd4d5a9421468cbf6a8cfcb30d09e3"
format_stamp: "Formatted at 2026-06-19 10:21:01 on dist-test-slave-6pt2"
server_key: "78e7443174ac9d6efe2884f48f64b7b8"
server_key_iv: "08d71c649bcde7befe09ee0a3ca72f8b"
server_key_version: "encryptionkey@0"
I20260619 10:21:01.658161 21175 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781864224577058-825-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781864224577058-825-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781864224577058-825-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 10:21:01.661275 21175 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 10:21:01.663030 21175 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 10:21:01.663129 21175 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 10:21:01.666564 21175 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:34381
I20260619 10:21:01.666581 21242 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:34381 every 8 connection(s)
I20260619 10:21:01.666949 21175 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781864224577058-825-0/minicluster-data/master-0/data/info.pb
I20260619 10:21:01.667840 21243 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 10:21:01.669054 825 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTU9OP6/build/release/bin/kudu as pid 21175
I20260619 10:21:01.669188 825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781864224577058-825-0/minicluster-data/master-0/wal/instance
I20260619 10:21:01.669353 825 external_mini_cluster.cc:1428] Setting key 52cd6e1b5e86b744d402aedea54e9d92
I20260619 10:21:01.670231 21243 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 14dd4d5a9421468cbf6a8cfcb30d09e3: Bootstrap starting.
I20260619 10:21:01.670876 21243 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 14dd4d5a9421468cbf6a8cfcb30d09e3: Neither blocks nor log segments found. Creating new log.
I20260619 10:21:01.671226 21243 log.cc:824] T 00000000000000000000000000000000 P 14dd4d5a9421468cbf6a8cfcb30d09e3: Log is configured to *not* fsync() on all Append() calls
I20260619 10:21:01.671855 21246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:21:01.669805 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33557 (local address 127.0.0.1:34381)
0619 10:21:01.669987 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0619 10:21:01.669996 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 10:21:01.670027 (+ 31us) server_negotiation.cc:407] Connection header received
0619 10:21:01.670549 (+ 522us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:21:01.670558 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:21:01.670593 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:21:01.670771 (+ 178us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:21:01.671377 (+ 606us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 10:21:01.671381 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 10:21:01.671381 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 10:21:01.671385 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0619 10:21:01.671421 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 10:21:01.671446 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 10:21:01.671519 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0619 10:21:01.671592 (+ 73us) server_negotiation.cc:299] Negotiation successful
0619 10:21:01.671658 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":133,"thread_start_us":87,"threads_started":1}
I20260619 10:21:01.672068 21243 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 14dd4d5a9421468cbf6a8cfcb30d09e3: No bootstrap required, opened a new log
I20260619 10:21:01.672711 825 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 10:21:01.674175 21243 raft_consensus.cc:348] T 00000000000000000000000000000000 P 14dd4d5a9421468cbf6a8cfcb30d09e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "14dd4d5a9421468cbf6a8cfcb30d09e3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34381 } }
I20260619 10:21:01.674345 21243 raft_consensus.cc:374] T 00000000000000000000000000000000 P 14dd4d5a9421468cbf6a8cfcb30d09e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 10:21:01.674417 21243 raft_consensus.cc:729] T 00000000000000000000000000000000 P 14dd4d5a9421468cbf6a8cfcb30d09e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 14dd4d5a9421468cbf6a8cfcb30d09e3, State: Initialized, Role: FOLLOWER
I20260619 10:21:01.674566 21243 consensus_queue.cc:260] T 00000000000000000000000000000000 P 14dd4d5a9421468cbf6a8cfcb30d09e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "14dd4d5a9421468cbf6a8cfcb30d09e3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34381 } }
I20260619 10:21:01.674680 21243 raft_consensus.cc:388] T 00000000000000000000000000000000 P 14dd4d5a9421468cbf6a8cfcb30d09e3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 10:21:01.674726 21243 raft_consensus.cc:482] T 00000000000000000000000000000000 P 14dd4d5a9421468cbf6a8cfcb30d09e3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 10:21:01.674770 21243 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 14dd4d5a9421468cbf6a8cfcb30d09e3 [term 0 FOLLOWER]: Advancing to term 1
I20260619 10:21:01.675187 21246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:21:01.674120 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35589 (local address 127.0.0.1:34381)
0619 10:21:01.674159 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0619 10:21:01.674161 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 10:21:01.674249 (+ 88us) server_negotiation.cc:407] Connection header received
0619 10:21:01.674302 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:21:01.674303 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:21:01.674307 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:21:01.674357 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:21:01.674989 (+ 632us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 10:21:01.674990 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 10:21:01.674991 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 10:21:01.674995 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0619 10:21:01.675007 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 10:21:01.675011 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 10:21:01.675077 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0619 10:21:01.675122 (+ 45us) server_negotiation.cc:299] Negotiation successful
0619 10:21:01.675134 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260619 10:21:01.675529 21243 raft_consensus.cc:504] T 00000000000000000000000000000000 P 14dd4d5a9421468cbf6a8cfcb30d09e3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "14dd4d5a9421468cbf6a8cfcb30d09e3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34381 } }
I20260619 10:21:01.675634 21243 leader_election.cc:302] T 00000000000000000000000000000000 P 14dd4d5a9421468cbf6a8cfcb30d09e3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 14dd4d5a9421468cbf6a8cfcb30d09e3; no voters:
I20260619 10:21:01.675783 21243 leader_election.cc:288] T 00000000000000000000000000000000 P 14dd4d5a9421468cbf6a8cfcb30d09e3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 10:21:01.675809 21253 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 14dd4d5a9421468cbf6a8cfcb30d09e3 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 10:21:01.676002 21253 raft_consensus.cc:686] T 00000000000000000000000000000000 P 14dd4d5a9421468cbf6a8cfcb30d09e3 [term 1 LEADER]: Becoming Leader. State: Replica: 14dd4d5a9421468cbf6a8cfcb30d09e3, State: Running, Role: LEADER
I20260619 10:21:01.676045 21243 sys_catalog.cc:564] T 00000000000000000000000000000000 P 14dd4d5a9421468cbf6a8cfcb30d09e3 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 10:21:01.676141 21253 consensus_queue.cc:237] T 00000000000000000000000000000000 P 14dd4d5a9421468cbf6a8cfcb30d09e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "14dd4d5a9421468cbf6a8cfcb30d09e3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34381 } }
I20260619 10:21:01.676470 21254 sys_catalog.cc:455] T 00000000000000000000000000000000 P 14dd4d5a9421468cbf6a8cfcb30d09e3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "14dd4d5a9421468cbf6a8cfcb30d09e3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "14dd4d5a9421468cbf6a8cfcb30d09e3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34381 } } }
I20260619 10:21:01.676478 21255 sys_catalog.cc:455] T 00000000000000000000000000000000 P 14dd4d5a9421468cbf6a8cfcb30d09e3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 14dd4d5a9421468cbf6a8cfcb30d09e3. Latest consensus state: current_term: 1 leader_uuid: "14dd4d5a9421468cbf6a8cfcb30d09e3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "14dd4d5a9421468cbf6a8cfcb30d09e3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34381 } } }
I20260619 10:21:01.676600 21254 sys_catalog.cc:458] T 00000000000000000000000000000000 P 14dd4d5a9421468cbf6a8cfcb30d09e3 [sys.catalog]: This master's current role is: LEADER
I20260619 10:21:01.676607 21255 sys_catalog.cc:458] T 00000000000000000000000000000000 P 14dd4d5a9421468cbf6a8cfcb30d09e3 [sys.catalog]: This master's current role is: LEADER
W20260619 10:21:01.677760 21269 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 14dd4d5a9421468cbf6a8cfcb30d09e3: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 10:21:01.677803 21269 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 10:21:01.677861 21262 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 10:21:01.678057 21262 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 10:21:01.679348 21262 catalog_manager.cc:1347] Generated new cluster ID: b0f583d2664a4be58a1085f8f957778a
I20260619 10:21:01.679395 21262 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 10:21:01.696985 21262 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 10:21:01.697165 21262 catalog_manager.cc:1504] Loading token signing keys...
I20260619 10:21:01.712294 21262 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 14dd4d5a9421468cbf6a8cfcb30d09e3: Generated new TSK 0
I20260619 10:21:01.712493 21262 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 10:21:01.717116 825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTU9OP6/build/release/bin/kudu with pid 21175
2026-06-19T10:21:01Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (1270 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781864224577058-825-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 10:21:01 dist-test-slave-6pt2 krb5kdc[21274](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 10:21:01 dist-test-slave-6pt2 krb5kdc[21274](info): set up 2 sockets
Jun 19 10:21:01 dist-test-slave-6pt2 krb5kdc[21274](info): commencing operation
krb5kdc: starting...
W20260619 10:21:02.726541 825 mini_kdc.cc:121] Time spent starting KDC: real 0.992s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781864224577058-825-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781864224577058-825-0/krb5kdc/test-user.keytab.
Jun 19 10:21:02 dist-test-slave-6pt2 krb5kdc[21274](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864462, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T10:21:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T10:21:02Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781864224577058-825-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781864224577058-825-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260619 10:21:02.833086 825 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTU9OP6/build/release/bin/kudu
/tmp/dist-test-taskTU9OP6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781864224577058-825-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781864224577058-825-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781864224577058-825-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781864224577058-825-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35761
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781864224577058-825-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:37067
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:35761
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781864224577058-825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781864224577058-825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781864224577058-825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 10:21:02.911080 21287 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 10:21:02.911309 21287 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 10:21:02.911348 21287 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 10:21:02.912460 21287 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 10:21:02.912525 21287 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 10:21:02.912547 21287 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 10:21:02.912568 21287 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 10:21:02.912597 21287 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 10:21:02.914048 21287 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:37067
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781864224577058-825-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781864224577058-825-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:35761
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781864224577058-825-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35761
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781864224577058-825-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21287
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781864224577058-825-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 09:32:44 UTC on ccycloud.quasar-gbtldp.root.comops.site
build id 4455440
I20260619 10:21:02.914319 21287 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 10:21:02.914542 21287 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 10:21:02.919303 21295 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 10:21:02.919337 21287 server_base.cc:1034] running on GCE node
W20260619 10:21:02.919281 21292 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 10:21:02.919284 21293 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 10:21:02.919713 21287 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 10:21:02.920002 21287 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 10:21:02.921169 21287 hybrid_clock.cc:648] HybridClock initialized: now 1781864462921163 us; error 35 us; skew 500 ppm
Jun 19 10:21:02 dist-test-slave-6pt2 krb5kdc[21274](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864462, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 10:21:02.923619 21287 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260619 10:21:02.924070 21287 webserver.cc:460] Webserver started at http://127.0.0.1:46091/ using document root <none> and password file <none>
I20260619 10:21:02.924283 21287 fs_manager.cc:359] Metadata directory not provided
I20260619 10:21:02.924348 21287 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 10:21:02.924499 21287 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 10:21:02.925526 21287 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781864224577058-825-0/minicluster-data/master-0/data/instance:
uuid: "8eb0712796e649d99bfb589063cbda32"
format_stamp: "Formatted at 2026-06-19 10:21:02 on dist-test-slave-6pt2"
server_key: "b62f5201e1e73b68e22aa15c6e0c970c"
server_key_iv: "3b63b41fc8411bbf5be4b7b467282700"
server_key_version: "encryptionkey@0"
I20260619 10:21:02.925866 21287 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781864224577058-825-0/minicluster-data/master-0/wal/instance:
uuid: "8eb0712796e649d99bfb589063cbda32"
format_stamp: "Formatted at 2026-06-19 10:21:02 on dist-test-slave-6pt2"
server_key: "b62f5201e1e73b68e22aa15c6e0c970c"
server_key_iv: "3b63b41fc8411bbf5be4b7b467282700"
server_key_version: "encryptionkey@0"
I20260619 10:21:02.927181 21287 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 10:21:02.927879 21302 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 10:21:02.928037 21287 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 10:21:02.928136 21287 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781864224577058-825-0/minicluster-data/master-0/data,/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781864224577058-825-0/minicluster-data/master-0/wal
uuid: "8eb0712796e649d99bfb589063cbda32"
format_stamp: "Formatted at 2026-06-19 10:21:02 on dist-test-slave-6pt2"
server_key: "b62f5201e1e73b68e22aa15c6e0c970c"
server_key_iv: "3b63b41fc8411bbf5be4b7b467282700"
server_key_version: "encryptionkey@0"
I20260619 10:21:02.928207 21287 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781864224577058-825-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781864224577058-825-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781864224577058-825-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 10:21:02.955067 21287 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 10:21:02.957437 21287 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 10:21:02.957604 21287 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 10:21:02.961575 21287 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:35761
I20260619 10:21:02.961613 21354 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:35761 every 8 connection(s)
I20260619 10:21:02.961984 21287 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781864224577058-825-0/minicluster-data/master-0/data/info.pb
I20260619 10:21:02.962850 21355 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 10:21:02.965117 21355 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8eb0712796e649d99bfb589063cbda32: Bootstrap starting.
I20260619 10:21:02.965678 21355 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8eb0712796e649d99bfb589063cbda32: Neither blocks nor log segments found. Creating new log.
I20260619 10:21:02.965961 21355 log.cc:824] T 00000000000000000000000000000000 P 8eb0712796e649d99bfb589063cbda32: Log is configured to *not* fsync() on all Append() calls
I20260619 10:21:02.966684 21355 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8eb0712796e649d99bfb589063cbda32: No bootstrap required, opened a new log
I20260619 10:21:02.967470 825 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTU9OP6/build/release/bin/kudu as pid 21287
I20260619 10:21:02.967562 825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781864224577058-825-0/minicluster-data/master-0/wal/instance
I20260619 10:21:02.967720 825 external_mini_cluster.cc:1428] Setting key 9c05782bcbcd1142c8008b764426bd26
I20260619 10:21:02.968822 21355 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8eb0712796e649d99bfb589063cbda32 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8eb0712796e649d99bfb589063cbda32" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35761 } }
I20260619 10:21:02.968997 21355 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8eb0712796e649d99bfb589063cbda32 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 10:21:02.969033 21355 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8eb0712796e649d99bfb589063cbda32 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8eb0712796e649d99bfb589063cbda32, State: Initialized, Role: FOLLOWER
I20260619 10:21:02.969224 21355 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8eb0712796e649d99bfb589063cbda32 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8eb0712796e649d99bfb589063cbda32" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35761 } }
I20260619 10:21:02.969290 21355 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8eb0712796e649d99bfb589063cbda32 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 10:21:02.969310 21355 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8eb0712796e649d99bfb589063cbda32 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 10:21:02.969375 21355 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8eb0712796e649d99bfb589063cbda32 [term 0 FOLLOWER]: Advancing to term 1
I20260619 10:21:02.970395 21355 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8eb0712796e649d99bfb589063cbda32 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8eb0712796e649d99bfb589063cbda32" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35761 } }
I20260619 10:21:02.970492 21355 leader_election.cc:302] T 00000000000000000000000000000000 P 8eb0712796e649d99bfb589063cbda32 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8eb0712796e649d99bfb589063cbda32; no voters:
I20260619 10:21:02.970643 21355 leader_election.cc:288] T 00000000000000000000000000000000 P 8eb0712796e649d99bfb589063cbda32 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 10:21:02.970695 21360 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8eb0712796e649d99bfb589063cbda32 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 10:21:02.970853 21355 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8eb0712796e649d99bfb589063cbda32 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 10:21:02.970857 21360 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8eb0712796e649d99bfb589063cbda32 [term 1 LEADER]: Becoming Leader. State: Replica: 8eb0712796e649d99bfb589063cbda32, State: Running, Role: LEADER
I20260619 10:21:02.971043 21360 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8eb0712796e649d99bfb589063cbda32 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8eb0712796e649d99bfb589063cbda32" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35761 } }
I20260619 10:21:02.972203 21361 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8eb0712796e649d99bfb589063cbda32 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8eb0712796e649d99bfb589063cbda32" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8eb0712796e649d99bfb589063cbda32" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35761 } } }
I20260619 10:21:02.972450 21361 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8eb0712796e649d99bfb589063cbda32 [sys.catalog]: This master's current role is: LEADER
W20260619 10:21:02.972584 21374 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 8eb0712796e649d99bfb589063cbda32: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 10:21:02.972692 21374 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 10:21:02.972389 21362 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8eb0712796e649d99bfb589063cbda32 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8eb0712796e649d99bfb589063cbda32. Latest consensus state: current_term: 1 leader_uuid: "8eb0712796e649d99bfb589063cbda32" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8eb0712796e649d99bfb589063cbda32" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35761 } } }
I20260619 10:21:02.972815 21362 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8eb0712796e649d99bfb589063cbda32 [sys.catalog]: This master's current role is: LEADER
I20260619 10:21:02.972977 21376 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 10:21:02.973630 21376 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Jun 19 10:21:02 dist-test-slave-6pt2 krb5kdc[21274](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864462, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260619 10:21:02.975687 21376 catalog_manager.cc:1347] Generated new cluster ID: e5629398139e4b318979ec5bac6b034f
I20260619 10:21:02.975747 21376 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 10:21:02.979342 21359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:21:02.968235 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51101 (local address 127.0.0.1:35761)
0619 10:21:02.968424 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0619 10:21:02.968434 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0619 10:21:02.968476 (+ 42us) server_negotiation.cc:407] Connection header received
0619 10:21:02.968790 (+ 314us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:21:02.968798 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:21:02.968867 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:21:02.969015 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:21:02.969789 (+ 774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:21:02.971146 (+ 1357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:21:02.971990 (+ 844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:21:02.972639 (+ 649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:21:02.972743 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:21:02.975660 (+ 2917us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 10:21:02.975664 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 10:21:02.975664 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 10:21:02.975675 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0619 10:21:02.977611 (+ 1936us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:21:02.977957 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:21:02.977960 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:21:02.977961 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:21:02.978075 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:21:02.978236 (+ 161us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:21:02.978237 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:21:02.978237 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:21:02.978504 (+ 267us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 10:21:02.978681 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0619 10:21:02.979051 (+ 370us) server_negotiation.cc:299] Negotiation successful
0619 10:21:02.979165 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":138,"thread_start_us":82,"threads_started":1}
I20260619 10:21:02.980278 825 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 10:21:02.983225 21376 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 10:21:02.983915 21376 catalog_manager.cc:1504] Loading token signing keys...
I20260619 10:21:02.997079 21359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:21:02.990874 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46777 (local address 127.0.0.1:35761)
0619 10:21:02.990924 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0619 10:21:02.990926 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 10:21:02.990987 (+ 61us) server_negotiation.cc:407] Connection header received
0619 10:21:02.991060 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:21:02.991061 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:21:02.991070 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:21:02.991139 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:21:02.991671 (+ 532us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:21:02.992360 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:21:02.993063 (+ 703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:21:02.993294 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:21:02.993378 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:21:02.994559 (+ 1181us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 10:21:02.994560 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 10:21:02.994561 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 10:21:02.994567 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 10:21:02.996014 (+ 1447us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:21:02.996358 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:21:02.996359 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:21:02.996360 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:21:02.996414 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:21:02.996604 (+ 190us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:21:02.996605 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:21:02.996606 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:21:02.996709 (+ 103us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 10:21:02.996820 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0619 10:21:02.996905 (+ 85us) server_negotiation.cc:299] Negotiation successful
0619 10:21:02.996977 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260619 10:21:02.998651 21376 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8eb0712796e649d99bfb589063cbda32: Generated new TSK 0
I20260619 10:21:02.998814 21376 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 10:21:03.003650 825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTU9OP6/build/release/bin/kudu with pid 21287
2026-06-19T10:21:03Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (1286 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781864224577058-825-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 10:21:03 dist-test-slave-6pt2 krb5kdc[21386](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 10:21:03 dist-test-slave-6pt2 krb5kdc[21386](info): set up 2 sockets
Jun 19 10:21:03 dist-test-slave-6pt2 krb5kdc[21386](info): commencing operation
krb5kdc: starting...
W20260619 10:21:04.006747 825 mini_kdc.cc:121] Time spent starting KDC: real 0.986s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781864224577058-825-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781864224577058-825-0/krb5kdc/test-user.keytab.
Jun 19 10:21:04 dist-test-slave-6pt2 krb5kdc[21386](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864464, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T10:21:04Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T10:21:04Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781864224577058-825-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781864224577058-825-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260619 10:21:04.118520 825 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTU9OP6/build/release/bin/kudu
/tmp/dist-test-taskTU9OP6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781864224577058-825-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781864224577058-825-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781864224577058-825-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781864224577058-825-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35979
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781864224577058-825-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:46407
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:35979
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781864224577058-825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781864224577058-825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781864224577058-825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 10:21:04.195887 21399 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 10:21:04.196146 21399 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 10:21:04.196183 21399 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 10:21:04.197345 21399 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 10:21:04.197424 21399 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 10:21:04.197445 21399 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 10:21:04.197467 21399 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 10:21:04.197494 21399 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 10:21:04.198952 21399 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:46407
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781864224577058-825-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781864224577058-825-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:35979
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781864224577058-825-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35979
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781864224577058-825-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21399
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781864224577058-825-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 09:32:44 UTC on ccycloud.quasar-gbtldp.root.comops.site
build id 4455440
I20260619 10:21:04.199215 21399 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 10:21:04.199434 21399 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260619 10:21:04.204385 21399 server_base.cc:1034] running on GCE node
W20260619 10:21:04.204511 21405 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 10:21:04.204526 21407 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 10:21:04.204546 21404 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 10:21:04.204969 21399 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 10:21:04.205257 21399 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 10:21:04.206419 21399 hybrid_clock.cc:648] HybridClock initialized: now 1781864464206405 us; error 26 us; skew 500 ppm
Jun 19 10:21:04 dist-test-slave-6pt2 krb5kdc[21386](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864464, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 10:21:04.212086 21399 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260619 10:21:04.212564 21399 webserver.cc:460] Webserver started at http://127.0.0.1:32905/ using document root <none> and password file <none>
I20260619 10:21:04.212805 21399 fs_manager.cc:359] Metadata directory not provided
I20260619 10:21:04.212869 21399 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 10:21:04.213023 21399 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 10:21:04.214040 21399 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781864224577058-825-0/minicluster-data/master-0/data/instance:
uuid: "4b1fc4f5785a4b28a57c6bfaed047a7c"
format_stamp: "Formatted at 2026-06-19 10:21:04 on dist-test-slave-6pt2"
server_key: "5709b79533db00c0856152fb8f1cceeb"
server_key_iv: "7bdb4e04431fbc84cadc35a620cda733"
server_key_version: "encryptionkey@0"
I20260619 10:21:04.214421 21399 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781864224577058-825-0/minicluster-data/master-0/wal/instance:
uuid: "4b1fc4f5785a4b28a57c6bfaed047a7c"
format_stamp: "Formatted at 2026-06-19 10:21:04 on dist-test-slave-6pt2"
server_key: "5709b79533db00c0856152fb8f1cceeb"
server_key_iv: "7bdb4e04431fbc84cadc35a620cda733"
server_key_version: "encryptionkey@0"
I20260619 10:21:04.215782 21399 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260619 10:21:04.216511 21414 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 10:21:04.216671 21399 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260619 10:21:04.216775 21399 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781864224577058-825-0/minicluster-data/master-0/data,/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781864224577058-825-0/minicluster-data/master-0/wal
uuid: "4b1fc4f5785a4b28a57c6bfaed047a7c"
format_stamp: "Formatted at 2026-06-19 10:21:04 on dist-test-slave-6pt2"
server_key: "5709b79533db00c0856152fb8f1cceeb"
server_key_iv: "7bdb4e04431fbc84cadc35a620cda733"
server_key_version: "encryptionkey@0"
I20260619 10:21:04.216859 21399 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781864224577058-825-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781864224577058-825-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781864224577058-825-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 10:21:04.231458 21399 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 10:21:04.233457 21399 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 10:21:04.233620 21399 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 10:21:04.237711 21399 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:35979
I20260619 10:21:04.237746 21466 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:35979 every 8 connection(s)
I20260619 10:21:04.238095 21399 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781864224577058-825-0/minicluster-data/master-0/data/info.pb
I20260619 10:21:04.239024 21467 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 10:21:04.241215 21467 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4b1fc4f5785a4b28a57c6bfaed047a7c: Bootstrap starting.
I20260619 10:21:04.241808 21467 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4b1fc4f5785a4b28a57c6bfaed047a7c: Neither blocks nor log segments found. Creating new log.
I20260619 10:21:04.242091 21467 log.cc:824] T 00000000000000000000000000000000 P 4b1fc4f5785a4b28a57c6bfaed047a7c: Log is configured to *not* fsync() on all Append() calls
I20260619 10:21:04.242782 825 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTU9OP6/build/release/bin/kudu as pid 21399
I20260619 10:21:04.242852 21467 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4b1fc4f5785a4b28a57c6bfaed047a7c: No bootstrap required, opened a new log
I20260619 10:21:04.242892 825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781864224577058-825-0/minicluster-data/master-0/wal/instance
I20260619 10:21:04.243067 825 external_mini_cluster.cc:1428] Setting key 7d239dbf19f12aeaaf4b78d1a536e4c1
I20260619 10:21:04.245755 21467 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4b1fc4f5785a4b28a57c6bfaed047a7c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b1fc4f5785a4b28a57c6bfaed047a7c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35979 } }
I20260619 10:21:04.245926 21467 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4b1fc4f5785a4b28a57c6bfaed047a7c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 10:21:04.245956 21467 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4b1fc4f5785a4b28a57c6bfaed047a7c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4b1fc4f5785a4b28a57c6bfaed047a7c, State: Initialized, Role: FOLLOWER
I20260619 10:21:04.246065 21467 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4b1fc4f5785a4b28a57c6bfaed047a7c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b1fc4f5785a4b28a57c6bfaed047a7c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35979 } }
I20260619 10:21:04.246146 21467 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4b1fc4f5785a4b28a57c6bfaed047a7c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 10:21:04.246193 21467 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4b1fc4f5785a4b28a57c6bfaed047a7c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 10:21:04.246240 21467 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4b1fc4f5785a4b28a57c6bfaed047a7c [term 0 FOLLOWER]: Advancing to term 1
I20260619 10:21:04.246853 21467 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4b1fc4f5785a4b28a57c6bfaed047a7c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b1fc4f5785a4b28a57c6bfaed047a7c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35979 } }
I20260619 10:21:04.246968 21467 leader_election.cc:302] T 00000000000000000000000000000000 P 4b1fc4f5785a4b28a57c6bfaed047a7c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4b1fc4f5785a4b28a57c6bfaed047a7c; no voters:
I20260619 10:21:04.247161 21467 leader_election.cc:288] T 00000000000000000000000000000000 P 4b1fc4f5785a4b28a57c6bfaed047a7c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 10:21:04.247187 21472 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4b1fc4f5785a4b28a57c6bfaed047a7c [term 1 FOLLOWER]: Leader election won for term 1
I20260619 10:21:04.247291 21472 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4b1fc4f5785a4b28a57c6bfaed047a7c [term 1 LEADER]: Becoming Leader. State: Replica: 4b1fc4f5785a4b28a57c6bfaed047a7c, State: Running, Role: LEADER
I20260619 10:21:04.247423 21472 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4b1fc4f5785a4b28a57c6bfaed047a7c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b1fc4f5785a4b28a57c6bfaed047a7c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35979 } }
I20260619 10:21:04.247434 21467 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4b1fc4f5785a4b28a57c6bfaed047a7c [sys.catalog]: configured and running, proceeding with master startup.
I20260619 10:21:04.247991 21474 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4b1fc4f5785a4b28a57c6bfaed047a7c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4b1fc4f5785a4b28a57c6bfaed047a7c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b1fc4f5785a4b28a57c6bfaed047a7c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35979 } } }
I20260619 10:21:04.248066 21474 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4b1fc4f5785a4b28a57c6bfaed047a7c [sys.catalog]: This master's current role is: LEADER
I20260619 10:21:04.248227 21473 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4b1fc4f5785a4b28a57c6bfaed047a7c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4b1fc4f5785a4b28a57c6bfaed047a7c. Latest consensus state: current_term: 1 leader_uuid: "4b1fc4f5785a4b28a57c6bfaed047a7c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b1fc4f5785a4b28a57c6bfaed047a7c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35979 } } }
I20260619 10:21:04.248293 21473 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4b1fc4f5785a4b28a57c6bfaed047a7c [sys.catalog]: This master's current role is: LEADER
I20260619 10:21:04.249107 21471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:21:04.243599 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46621 (local address 127.0.0.1:35979)
0619 10:21:04.243782 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0619 10:21:04.243791 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 10:21:04.243823 (+ 32us) server_negotiation.cc:407] Connection header received
0619 10:21:04.244266 (+ 443us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:21:04.244275 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:21:04.244336 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:21:04.244487 (+ 151us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:21:04.244659 (+ 172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:21:04.246078 (+ 1419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:21:04.246981 (+ 903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:21:04.247579 (+ 598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:21:04.248001 (+ 422us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:21:04.248414 (+ 413us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 10:21:04.248417 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 10:21:04.248418 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 10:21:04.248629 (+ 211us) server_negotiation.cc:922] calling sasl_server_start()
0619 10:21:04.248665 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 10:21:04.248679 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 10:21:04.248743 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0619 10:21:04.248835 (+ 92us) server_negotiation.cc:299] Negotiation successful
0619 10:21:04.248881 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":131,"thread_start_us":80,"threads_started":1}
W20260619 10:21:04.249778 21488 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 4b1fc4f5785a4b28a57c6bfaed047a7c: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 10:21:04.249826 21488 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 10:21:04.249923 21476 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 10:21:04.250080 21476 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 10:21:04.250113 825 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 10:21:04.252332 21476 catalog_manager.cc:1347] Generated new cluster ID: fa794be94f514c628bce202ae6686cec
I20260619 10:21:04.252382 21476 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 10:21:04.254984 21471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:21:04.251615 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46553 (local address 127.0.0.1:35979)
0619 10:21:04.251653 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0619 10:21:04.251655 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 10:21:04.251751 (+ 96us) server_negotiation.cc:407] Connection header received
0619 10:21:04.251797 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:21:04.251798 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:21:04.251810 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:21:04.251871 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:21:04.251997 (+ 126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:21:04.252676 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:21:04.253703 (+ 1027us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:21:04.253992 (+ 289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:21:04.254047 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:21:04.254746 (+ 699us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 10:21:04.254747 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 10:21:04.254748 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 10:21:04.254753 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0619 10:21:04.254767 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 10:21:04.254771 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 10:21:04.254820 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0619 10:21:04.254894 (+ 74us) server_negotiation.cc:299] Negotiation successful
0619 10:21:04.254912 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260619 10:21:04.268271 21476 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 10:21:04.269039 21476 catalog_manager.cc:1504] Loading token signing keys...
I20260619 10:21:04.276044 21476 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4b1fc4f5785a4b28a57c6bfaed047a7c: Generated new TSK 0
I20260619 10:21:04.276226 21476 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 10:21:04.280122 825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTU9OP6/build/release/bin/kudu with pid 21399
2026-06-19T10:21:04Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (1276 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781864224577058-825-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 10:21:04 dist-test-slave-6pt2 krb5kdc[21498](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 10:21:04 dist-test-slave-6pt2 krb5kdc[21498](info): set up 2 sockets
Jun 19 10:21:04 dist-test-slave-6pt2 krb5kdc[21498](info): commencing operation
krb5kdc: starting...
W20260619 10:21:05.298722 825 mini_kdc.cc:121] Time spent starting KDC: real 1.001s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781864224577058-825-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781864224577058-825-0/krb5kdc/test-user.keytab.
Jun 19 10:21:05 dist-test-slave-6pt2 krb5kdc[21498](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864465, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T10:21:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T10:21:05Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781864224577058-825-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781864224577058-825-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260619 10:21:05.404889 825 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTU9OP6/build/release/bin/kudu
/tmp/dist-test-taskTU9OP6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781864224577058-825-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781864224577058-825-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781864224577058-825-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781864224577058-825-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34147
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781864224577058-825-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:45663
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:34147
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781864224577058-825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781864224577058-825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781864224577058-825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 10:21:05.481715 21511 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 10:21:05.481894 21511 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 10:21:05.481915 21511 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 10:21:05.483022 21511 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 10:21:05.483062 21511 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 10:21:05.483076 21511 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 10:21:05.483089 21511 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 10:21:05.483101 21511 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 10:21:05.484467 21511 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:45663
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781864224577058-825-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781864224577058-825-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:34147
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781864224577058-825-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34147
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781864224577058-825-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21511
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781864224577058-825-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 09:32:44 UTC on ccycloud.quasar-gbtldp.root.comops.site
build id 4455440
I20260619 10:21:05.484642 21511 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 10:21:05.484802 21511 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 10:21:05.489324 21517 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 10:21:05.489336 21516 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 10:21:05.489415 21519 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 10:21:05.489686 21511 server_base.cc:1034] running on GCE node
I20260619 10:21:05.489941 21511 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 10:21:05.490219 21511 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 10:21:05.491353 21511 hybrid_clock.cc:648] HybridClock initialized: now 1781864465491331 us; error 36 us; skew 500 ppm
Jun 19 10:21:05 dist-test-slave-6pt2 krb5kdc[21498](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864465, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 10:21:05.493721 21511 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260619 10:21:05.494120 21511 webserver.cc:460] Webserver started at http://127.0.0.1:44289/ using document root <none> and password file <none>
I20260619 10:21:05.494351 21511 fs_manager.cc:359] Metadata directory not provided
I20260619 10:21:05.494395 21511 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 10:21:05.494554 21511 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 10:21:05.495559 21511 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781864224577058-825-0/minicluster-data/master-0/data/instance:
uuid: "19e2868187714a3c8363ad03f23c239e"
format_stamp: "Formatted at 2026-06-19 10:21:05 on dist-test-slave-6pt2"
server_key: "1b9af6d1ac8a202253a47327e0da0a8f"
server_key_iv: "62b83d3a8f4b239a2b048564aeddb9e4"
server_key_version: "encryptionkey@0"
I20260619 10:21:05.495890 21511 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781864224577058-825-0/minicluster-data/master-0/wal/instance:
uuid: "19e2868187714a3c8363ad03f23c239e"
format_stamp: "Formatted at 2026-06-19 10:21:05 on dist-test-slave-6pt2"
server_key: "1b9af6d1ac8a202253a47327e0da0a8f"
server_key_iv: "62b83d3a8f4b239a2b048564aeddb9e4"
server_key_version: "encryptionkey@0"
I20260619 10:21:05.497174 21511 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 10:21:05.497825 21526 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 10:21:05.498014 21511 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 10:21:05.498114 21511 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781864224577058-825-0/minicluster-data/master-0/data,/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781864224577058-825-0/minicluster-data/master-0/wal
uuid: "19e2868187714a3c8363ad03f23c239e"
format_stamp: "Formatted at 2026-06-19 10:21:05 on dist-test-slave-6pt2"
server_key: "1b9af6d1ac8a202253a47327e0da0a8f"
server_key_iv: "62b83d3a8f4b239a2b048564aeddb9e4"
server_key_version: "encryptionkey@0"
I20260619 10:21:05.498188 21511 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781864224577058-825-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781864224577058-825-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781864224577058-825-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 10:21:05.501408 21511 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 10:21:05.503185 21511 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 10:21:05.503327 21511 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 10:21:05.506934 21511 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:34147
I20260619 10:21:05.506961 21578 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:34147 every 8 connection(s)
I20260619 10:21:05.507285 21511 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781864224577058-825-0/minicluster-data/master-0/data/info.pb
I20260619 10:21:05.508229 21579 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 10:21:05.509871 825 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTU9OP6/build/release/bin/kudu as pid 21511
I20260619 10:21:05.509989 825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781864224577058-825-0/minicluster-data/master-0/wal/instance
I20260619 10:21:05.510164 825 external_mini_cluster.cc:1428] Setting key 31b0dcfb86a00a08798e590dcaf020a5
I20260619 10:21:05.510440 21579 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 19e2868187714a3c8363ad03f23c239e: Bootstrap starting.
I20260619 10:21:05.511173 21579 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 19e2868187714a3c8363ad03f23c239e: Neither blocks nor log segments found. Creating new log.
I20260619 10:21:05.511476 21579 log.cc:824] T 00000000000000000000000000000000 P 19e2868187714a3c8363ad03f23c239e: Log is configured to *not* fsync() on all Append() calls
I20260619 10:21:05.512188 21579 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 19e2868187714a3c8363ad03f23c239e: No bootstrap required, opened a new log
I20260619 10:21:05.512856 21582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:21:05.510690 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35079 (local address 127.0.0.1:34147)
0619 10:21:05.510875 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0619 10:21:05.510884 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 10:21:05.510921 (+ 37us) server_negotiation.cc:407] Connection header received
0619 10:21:05.511539 (+ 618us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:21:05.511548 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:21:05.511579 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:21:05.511726 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:21:05.512357 (+ 631us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 10:21:05.512361 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 10:21:05.512361 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 10:21:05.512365 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0619 10:21:05.512410 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 10:21:05.512424 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 10:21:05.512515 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0619 10:21:05.512632 (+ 117us) server_negotiation.cc:299] Negotiation successful
0619 10:21:05.512672 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":128,"thread_start_us":73,"threads_started":1}
I20260619 10:21:05.513809 825 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 10:21:05.514468 21579 raft_consensus.cc:348] T 00000000000000000000000000000000 P 19e2868187714a3c8363ad03f23c239e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "19e2868187714a3c8363ad03f23c239e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34147 } }
I20260619 10:21:05.514612 21579 raft_consensus.cc:374] T 00000000000000000000000000000000 P 19e2868187714a3c8363ad03f23c239e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 10:21:05.514689 21579 raft_consensus.cc:729] T 00000000000000000000000000000000 P 19e2868187714a3c8363ad03f23c239e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 19e2868187714a3c8363ad03f23c239e, State: Initialized, Role: FOLLOWER
I20260619 10:21:05.514810 21579 consensus_queue.cc:260] T 00000000000000000000000000000000 P 19e2868187714a3c8363ad03f23c239e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "19e2868187714a3c8363ad03f23c239e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34147 } }
I20260619 10:21:05.514918 21579 raft_consensus.cc:388] T 00000000000000000000000000000000 P 19e2868187714a3c8363ad03f23c239e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 10:21:05.514964 21579 raft_consensus.cc:482] T 00000000000000000000000000000000 P 19e2868187714a3c8363ad03f23c239e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 10:21:05.515012 21579 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 19e2868187714a3c8363ad03f23c239e [term 0 FOLLOWER]: Advancing to term 1
I20260619 10:21:05.515640 21579 raft_consensus.cc:504] T 00000000000000000000000000000000 P 19e2868187714a3c8363ad03f23c239e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "19e2868187714a3c8363ad03f23c239e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34147 } }
I20260619 10:21:05.515748 21579 leader_election.cc:302] T 00000000000000000000000000000000 P 19e2868187714a3c8363ad03f23c239e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 19e2868187714a3c8363ad03f23c239e; no voters:
I20260619 10:21:05.515923 21579 leader_election.cc:288] T 00000000000000000000000000000000 P 19e2868187714a3c8363ad03f23c239e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 10:21:05.515956 21589 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 19e2868187714a3c8363ad03f23c239e [term 1 FOLLOWER]: Leader election won for term 1
I20260619 10:21:05.516160 21589 raft_consensus.cc:686] T 00000000000000000000000000000000 P 19e2868187714a3c8363ad03f23c239e [term 1 LEADER]: Becoming Leader. State: Replica: 19e2868187714a3c8363ad03f23c239e, State: Running, Role: LEADER
I20260619 10:21:05.516276 21589 consensus_queue.cc:237] T 00000000000000000000000000000000 P 19e2868187714a3c8363ad03f23c239e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "19e2868187714a3c8363ad03f23c239e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34147 } }
I20260619 10:21:05.516357 21582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:21:05.515130 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48379 (local address 127.0.0.1:34147)
0619 10:21:05.515164 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0619 10:21:05.515166 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 10:21:05.515313 (+ 147us) server_negotiation.cc:407] Connection header received
0619 10:21:05.515353 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:21:05.515353 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:21:05.515358 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:21:05.515434 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:21:05.516002 (+ 568us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 10:21:05.516004 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 10:21:05.516004 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 10:21:05.516008 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0619 10:21:05.516021 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 10:21:05.516025 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 10:21:05.516084 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0619 10:21:05.516192 (+ 108us) server_negotiation.cc:299] Negotiation successful
0619 10:21:05.516204 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260619 10:21:05.516251 21579 sys_catalog.cc:564] T 00000000000000000000000000000000 P 19e2868187714a3c8363ad03f23c239e [sys.catalog]: configured and running, proceeding with master startup.
I20260619 10:21:05.516636 21590 sys_catalog.cc:455] T 00000000000000000000000000000000 P 19e2868187714a3c8363ad03f23c239e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "19e2868187714a3c8363ad03f23c239e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "19e2868187714a3c8363ad03f23c239e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34147 } } }
I20260619 10:21:05.516688 21591 sys_catalog.cc:455] T 00000000000000000000000000000000 P 19e2868187714a3c8363ad03f23c239e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 19e2868187714a3c8363ad03f23c239e. Latest consensus state: current_term: 1 leader_uuid: "19e2868187714a3c8363ad03f23c239e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "19e2868187714a3c8363ad03f23c239e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34147 } } }
I20260619 10:21:05.516772 21590 sys_catalog.cc:458] T 00000000000000000000000000000000 P 19e2868187714a3c8363ad03f23c239e [sys.catalog]: This master's current role is: LEADER
I20260619 10:21:05.516805 21591 sys_catalog.cc:458] T 00000000000000000000000000000000 P 19e2868187714a3c8363ad03f23c239e [sys.catalog]: This master's current role is: LEADER
I20260619 10:21:05.517009 21593 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 10:21:05.517542 21593 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 10:21:05.518898 21593 catalog_manager.cc:1347] Generated new cluster ID: 34fdf97e868d498ba124b93b36da0aaf
I20260619 10:21:05.518971 21593 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 10:21:05.542347 21593 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 10:21:05.542552 21593 catalog_manager.cc:1504] Loading token signing keys...
I20260619 10:21:05.556409 21593 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 19e2868187714a3c8363ad03f23c239e: Generated new TSK 0
I20260619 10:21:05.556629 21593 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 10:21:05.558734 825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTU9OP6/build/release/bin/kudu with pid 21511
2026-06-19T10:21:05Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (1278 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781864224577058-825-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 10:21:05 dist-test-slave-6pt2 krb5kdc[21610](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 10:21:05 dist-test-slave-6pt2 krb5kdc[21610](info): set up 2 sockets
Jun 19 10:21:05 dist-test-slave-6pt2 krb5kdc[21610](info): commencing operation
krb5kdc: starting...
W20260619 10:21:06.558564 825 mini_kdc.cc:121] Time spent starting KDC: real 0.982s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781864224577058-825-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781864224577058-825-0/krb5kdc/test-user.keytab.
Jun 19 10:21:06 dist-test-slave-6pt2 krb5kdc[21610](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864466, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T10:21:06Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T10:21:06Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781864224577058-825-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781864224577058-825-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260619 10:21:06.667260 825 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTU9OP6/build/release/bin/kudu
/tmp/dist-test-taskTU9OP6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781864224577058-825-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781864224577058-825-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781864224577058-825-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781864224577058-825-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35153
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781864224577058-825-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:37425
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:35153
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781864224577058-825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781864224577058-825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781864224577058-825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 10:21:06.744170 21623 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 10:21:06.744412 21623 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 10:21:06.744449 21623 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 10:21:06.745575 21623 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 10:21:06.745640 21623 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 10:21:06.745663 21623 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 10:21:06.745689 21623 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 10:21:06.745714 21623 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 10:21:06.747090 21623 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:37425
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781864224577058-825-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781864224577058-825-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:35153
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781864224577058-825-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35153
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781864224577058-825-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21623
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781864224577058-825-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 09:32:44 UTC on ccycloud.quasar-gbtldp.root.comops.site
build id 4455440
I20260619 10:21:06.747347 21623 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 10:21:06.747556 21623 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 10:21:06.751935 21628 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 10:21:06.752008 21631 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 10:21:06.752142 21623 server_base.cc:1034] running on GCE node
W20260619 10:21:06.752178 21629 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 10:21:06.752413 21623 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 10:21:06.752681 21623 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 10:21:06.753814 21623 hybrid_clock.cc:648] HybridClock initialized: now 1781864466753794 us; error 38 us; skew 500 ppm
Jun 19 10:21:06 dist-test-slave-6pt2 krb5kdc[21610](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864466, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 10:21:06.756174 21623 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260619 10:21:06.756517 21623 webserver.cc:460] Webserver started at http://127.0.0.1:46477/ using document root <none> and password file <none>
I20260619 10:21:06.756685 21623 fs_manager.cc:359] Metadata directory not provided
I20260619 10:21:06.756723 21623 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 10:21:06.756807 21623 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 10:21:06.757874 21623 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781864224577058-825-0/minicluster-data/master-0/data/instance:
uuid: "024c5f9731474efd87295823af6a4857"
format_stamp: "Formatted at 2026-06-19 10:21:06 on dist-test-slave-6pt2"
server_key: "25b79c022e0f07fa5b266bf96d83c064"
server_key_iv: "c266dbc9c57ed38f001dbe6d8c9a2980"
server_key_version: "encryptionkey@0"
I20260619 10:21:06.758172 21623 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781864224577058-825-0/minicluster-data/master-0/wal/instance:
uuid: "024c5f9731474efd87295823af6a4857"
format_stamp: "Formatted at 2026-06-19 10:21:06 on dist-test-slave-6pt2"
server_key: "25b79c022e0f07fa5b266bf96d83c064"
server_key_iv: "c266dbc9c57ed38f001dbe6d8c9a2980"
server_key_version: "encryptionkey@0"
I20260619 10:21:06.759529 21623 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 10:21:06.760166 21638 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 10:21:06.760371 21623 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 10:21:06.760473 21623 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781864224577058-825-0/minicluster-data/master-0/data,/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781864224577058-825-0/minicluster-data/master-0/wal
uuid: "024c5f9731474efd87295823af6a4857"
format_stamp: "Formatted at 2026-06-19 10:21:06 on dist-test-slave-6pt2"
server_key: "25b79c022e0f07fa5b266bf96d83c064"
server_key_iv: "c266dbc9c57ed38f001dbe6d8c9a2980"
server_key_version: "encryptionkey@0"
I20260619 10:21:06.760556 21623 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781864224577058-825-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781864224577058-825-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781864224577058-825-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 10:21:06.781364 21623 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 10:21:06.783120 21623 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 10:21:06.783277 21623 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 10:21:06.787039 21623 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:35153
I20260619 10:21:06.787087 21690 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:35153 every 8 connection(s)
I20260619 10:21:06.787433 21623 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781864224577058-825-0/minicluster-data/master-0/data/info.pb
I20260619 10:21:06.788235 21691 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 10:21:06.790299 21691 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 024c5f9731474efd87295823af6a4857: Bootstrap starting.
I20260619 10:21:06.790892 21691 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 024c5f9731474efd87295823af6a4857: Neither blocks nor log segments found. Creating new log.
I20260619 10:21:06.791172 21691 log.cc:824] T 00000000000000000000000000000000 P 024c5f9731474efd87295823af6a4857: Log is configured to *not* fsync() on all Append() calls
I20260619 10:21:06.791635 825 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTU9OP6/build/release/bin/kudu as pid 21623
I20260619 10:21:06.791772 825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781864224577058-825-0/minicluster-data/master-0/wal/instance
I20260619 10:21:06.791952 825 external_mini_cluster.cc:1428] Setting key 0f9db62804252dd0710c41d347a9ea4e
I20260619 10:21:06.791972 21691 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 024c5f9731474efd87295823af6a4857: No bootstrap required, opened a new log
I20260619 10:21:06.793820 21691 raft_consensus.cc:348] T 00000000000000000000000000000000 P 024c5f9731474efd87295823af6a4857 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "024c5f9731474efd87295823af6a4857" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35153 } }
I20260619 10:21:06.793946 21691 raft_consensus.cc:374] T 00000000000000000000000000000000 P 024c5f9731474efd87295823af6a4857 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 10:21:06.793974 21691 raft_consensus.cc:729] T 00000000000000000000000000000000 P 024c5f9731474efd87295823af6a4857 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 024c5f9731474efd87295823af6a4857, State: Initialized, Role: FOLLOWER
I20260619 10:21:06.794145 21691 consensus_queue.cc:260] T 00000000000000000000000000000000 P 024c5f9731474efd87295823af6a4857 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "024c5f9731474efd87295823af6a4857" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35153 } }
I20260619 10:21:06.794231 21691 raft_consensus.cc:388] T 00000000000000000000000000000000 P 024c5f9731474efd87295823af6a4857 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 10:21:06.794275 21691 raft_consensus.cc:482] T 00000000000000000000000000000000 P 024c5f9731474efd87295823af6a4857 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 10:21:06.794327 21691 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 024c5f9731474efd87295823af6a4857 [term 0 FOLLOWER]: Advancing to term 1
I20260619 10:21:06.795145 21691 raft_consensus.cc:504] T 00000000000000000000000000000000 P 024c5f9731474efd87295823af6a4857 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "024c5f9731474efd87295823af6a4857" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35153 } }
I20260619 10:21:06.795258 21691 leader_election.cc:302] T 00000000000000000000000000000000 P 024c5f9731474efd87295823af6a4857 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 024c5f9731474efd87295823af6a4857; no voters:
I20260619 10:21:06.795440 21691 leader_election.cc:288] T 00000000000000000000000000000000 P 024c5f9731474efd87295823af6a4857 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 10:21:06.795480 21696 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 024c5f9731474efd87295823af6a4857 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 10:21:06.795594 21696 raft_consensus.cc:686] T 00000000000000000000000000000000 P 024c5f9731474efd87295823af6a4857 [term 1 LEADER]: Becoming Leader. State: Replica: 024c5f9731474efd87295823af6a4857, State: Running, Role: LEADER
I20260619 10:21:06.795697 21691 sys_catalog.cc:564] T 00000000000000000000000000000000 P 024c5f9731474efd87295823af6a4857 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 10:21:06.795745 21696 consensus_queue.cc:237] T 00000000000000000000000000000000 P 024c5f9731474efd87295823af6a4857 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "024c5f9731474efd87295823af6a4857" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35153 } }
I20260619 10:21:06.796070 21696 sys_catalog.cc:455] T 00000000000000000000000000000000 P 024c5f9731474efd87295823af6a4857 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 024c5f9731474efd87295823af6a4857. Latest consensus state: current_term: 1 leader_uuid: "024c5f9731474efd87295823af6a4857" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "024c5f9731474efd87295823af6a4857" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35153 } } }
I20260619 10:21:06.796082 21697 sys_catalog.cc:455] T 00000000000000000000000000000000 P 024c5f9731474efd87295823af6a4857 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "024c5f9731474efd87295823af6a4857" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "024c5f9731474efd87295823af6a4857" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35153 } } }
I20260619 10:21:06.796309 21697 sys_catalog.cc:458] T 00000000000000000000000000000000 P 024c5f9731474efd87295823af6a4857 [sys.catalog]: This master's current role is: LEADER
I20260619 10:21:06.796229 21696 sys_catalog.cc:458] T 00000000000000000000000000000000 P 024c5f9731474efd87295823af6a4857 [sys.catalog]: This master's current role is: LEADER
W20260619 10:21:06.797497 21711 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 024c5f9731474efd87295823af6a4857: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 10:21:06.797580 21711 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 10:21:06.797680 21705 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 10:21:06.797869 21705 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 10:21:06.799340 21705 catalog_manager.cc:1347] Generated new cluster ID: f3304111cf264d1aa8864d80a27fc3d4
I20260619 10:21:06.799391 21705 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Jun 19 10:21:06 dist-test-slave-6pt2 krb5kdc[21610](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864466, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260619 10:21:06.805258 21695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:21:06.792419 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.235:56003 (local address 127.0.0.1:35153)
0619 10:21:06.792580 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0619 10:21:06.792589 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 10:21:06.792643 (+ 54us) server_negotiation.cc:407] Connection header received
0619 10:21:06.793822 (+ 1179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:21:06.793831 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:21:06.793889 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:21:06.794066 (+ 177us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:21:06.794905 (+ 839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:21:06.796277 (+ 1372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:21:06.797474 (+ 1197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:21:06.798049 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:21:06.798158 (+ 109us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:21:06.801257 (+ 3099us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 10:21:06.801261 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 10:21:06.801261 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 10:21:06.801273 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0619 10:21:06.803325 (+ 2052us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:21:06.803832 (+ 507us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:21:06.803835 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:21:06.803835 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:21:06.803962 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:21:06.804202 (+ 240us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:21:06.804203 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:21:06.804203 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:21:06.804523 (+ 320us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 10:21:06.804686 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0619 10:21:06.804820 (+ 134us) server_negotiation.cc:299] Negotiation successful
0619 10:21:06.805043 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":117,"thread_start_us":77,"threads_started":1}
I20260619 10:21:06.806155 825 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 10:21:06.810316 21705 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 10:21:06.810885 21705 catalog_manager.cc:1504] Loading token signing keys...
I20260619 10:21:06.815088 21695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:21:06.807712 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.235:34005 (local address 127.0.0.1:35153)
0619 10:21:06.807750 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0619 10:21:06.807752 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 10:21:06.807817 (+ 65us) server_negotiation.cc:407] Connection header received
0619 10:21:06.807887 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:21:06.807888 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:21:06.807898 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:21:06.807951 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:21:06.808546 (+ 595us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:21:06.809300 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:21:06.810282 (+ 982us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:21:06.810627 (+ 345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:21:06.810666 (+ 39us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:21:06.812180 (+ 1514us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 10:21:06.812182 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 10:21:06.812182 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 10:21:06.812189 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 10:21:06.813800 (+ 1611us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:21:06.814202 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:21:06.814203 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:21:06.814204 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:21:06.814256 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:21:06.814517 (+ 261us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:21:06.814518 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:21:06.814518 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:21:06.814676 (+ 158us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 10:21:06.814777 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0619 10:21:06.814936 (+ 159us) server_negotiation.cc:299] Negotiation successful
0619 10:21:06.814997 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260619 10:21:06.816805 21705 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 024c5f9731474efd87295823af6a4857: Generated new TSK 0
I20260619 10:21:06.816995 21705 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 10:21:06.821758 825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTU9OP6/build/release/bin/kudu with pid 21623
W20260619 10:21:06.825966 21608 connection.cc:535] client connection to 127.0.0.1:35153 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:35153): Connection reset by peer (error 104)
2026-06-19T10:21:06Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (1263 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781864224577058-825-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 10:21:06 dist-test-slave-6pt2 krb5kdc[21722](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 10:21:06 dist-test-slave-6pt2 krb5kdc[21722](info): set up 2 sockets
Jun 19 10:21:06 dist-test-slave-6pt2 krb5kdc[21722](info): commencing operation
krb5kdc: starting...
W20260619 10:21:07.822453 825 mini_kdc.cc:121] Time spent starting KDC: real 0.983s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781864224577058-825-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781864224577058-825-0/krb5kdc/test-user.keytab.
Jun 19 10:21:07 dist-test-slave-6pt2 krb5kdc[21722](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864467, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T10:21:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T10:21:07Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781864224577058-825-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781864224577058-825-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260619 10:21:07.928845 825 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTU9OP6/build/release/bin/kudu
/tmp/dist-test-taskTU9OP6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781864224577058-825-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781864224577058-825-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781864224577058-825-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781864224577058-825-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36491
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781864224577058-825-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:39977
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:36491
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781864224577058-825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781864224577058-825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781864224577058-825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 10:21:08.006049 21735 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 10:21:08.006266 21735 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 10:21:08.006300 21735 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 10:21:08.007444 21735 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 10:21:08.007515 21735 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 10:21:08.007539 21735 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 10:21:08.007563 21735 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 10:21:08.007593 21735 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 10:21:08.009110 21735 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:39977
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781864224577058-825-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781864224577058-825-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:36491
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781864224577058-825-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36491
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781864224577058-825-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21735
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781864224577058-825-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 09:32:44 UTC on ccycloud.quasar-gbtldp.root.comops.site
build id 4455440
I20260619 10:21:08.009362 21735 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 10:21:08.009569 21735 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 10:21:08.014101 21741 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 10:21:08.014082 21743 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 10:21:08.014264 21735 server_base.cc:1034] running on GCE node
W20260619 10:21:08.014117 21740 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 10:21:08.014551 21735 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 10:21:08.014860 21735 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 10:21:08.016006 21735 hybrid_clock.cc:648] HybridClock initialized: now 1781864468015989 us; error 27 us; skew 500 ppm
Jun 19 10:21:08 dist-test-slave-6pt2 krb5kdc[21722](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864468, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 10:21:08.018323 21735 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260619 10:21:08.018765 21735 webserver.cc:460] Webserver started at http://127.0.0.1:41627/ using document root <none> and password file <none>
I20260619 10:21:08.018982 21735 fs_manager.cc:359] Metadata directory not provided
I20260619 10:21:08.019045 21735 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 10:21:08.019171 21735 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 10:21:08.020112 21735 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781864224577058-825-0/minicluster-data/master-0/data/instance:
uuid: "face107a2b714ce888198f167e82a4b4"
format_stamp: "Formatted at 2026-06-19 10:21:08 on dist-test-slave-6pt2"
server_key: "65e353a4ecd8a7970b12421b584c9a5a"
server_key_iv: "3ac799db438c4def2aee1c7e2942858e"
server_key_version: "encryptionkey@0"
I20260619 10:21:08.020448 21735 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781864224577058-825-0/minicluster-data/master-0/wal/instance:
uuid: "face107a2b714ce888198f167e82a4b4"
format_stamp: "Formatted at 2026-06-19 10:21:08 on dist-test-slave-6pt2"
server_key: "65e353a4ecd8a7970b12421b584c9a5a"
server_key_iv: "3ac799db438c4def2aee1c7e2942858e"
server_key_version: "encryptionkey@0"
I20260619 10:21:08.022490 21735 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.003s
I20260619 10:21:08.023250 21750 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 10:21:08.023420 21735 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260619 10:21:08.023514 21735 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781864224577058-825-0/minicluster-data/master-0/data,/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781864224577058-825-0/minicluster-data/master-0/wal
uuid: "face107a2b714ce888198f167e82a4b4"
format_stamp: "Formatted at 2026-06-19 10:21:08 on dist-test-slave-6pt2"
server_key: "65e353a4ecd8a7970b12421b584c9a5a"
server_key_iv: "3ac799db438c4def2aee1c7e2942858e"
server_key_version: "encryptionkey@0"
I20260619 10:21:08.023593 21735 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781864224577058-825-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781864224577058-825-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781864224577058-825-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 10:21:08.033001 21735 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 10:21:08.034708 21735 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 10:21:08.034852 21735 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 10:21:08.038465 21735 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:36491
I20260619 10:21:08.038514 21802 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:36491 every 8 connection(s)
I20260619 10:21:08.038894 21735 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781864224577058-825-0/minicluster-data/master-0/data/info.pb
I20260619 10:21:08.039673 21803 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 10:21:08.041692 21803 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P face107a2b714ce888198f167e82a4b4: Bootstrap starting.
I20260619 10:21:08.042156 21803 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P face107a2b714ce888198f167e82a4b4: Neither blocks nor log segments found. Creating new log.
I20260619 10:21:08.042373 21803 log.cc:824] T 00000000000000000000000000000000 P face107a2b714ce888198f167e82a4b4: Log is configured to *not* fsync() on all Append() calls
I20260619 10:21:08.043008 21803 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P face107a2b714ce888198f167e82a4b4: No bootstrap required, opened a new log
I20260619 10:21:08.043084 825 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTU9OP6/build/release/bin/kudu as pid 21735
I20260619 10:21:08.043226 825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781864224577058-825-0/minicluster-data/master-0/wal/instance
I20260619 10:21:08.043422 825 external_mini_cluster.cc:1428] Setting key 4fc9798ec6f28dbd213868317266b070
I20260619 10:21:08.045193 21803 raft_consensus.cc:348] T 00000000000000000000000000000000 P face107a2b714ce888198f167e82a4b4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "face107a2b714ce888198f167e82a4b4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36491 } }
I20260619 10:21:08.045383 21803 raft_consensus.cc:374] T 00000000000000000000000000000000 P face107a2b714ce888198f167e82a4b4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 10:21:08.045418 21803 raft_consensus.cc:729] T 00000000000000000000000000000000 P face107a2b714ce888198f167e82a4b4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: face107a2b714ce888198f167e82a4b4, State: Initialized, Role: FOLLOWER
I20260619 10:21:08.045547 21803 consensus_queue.cc:260] T 00000000000000000000000000000000 P face107a2b714ce888198f167e82a4b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "face107a2b714ce888198f167e82a4b4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36491 } }
I20260619 10:21:08.045653 21803 raft_consensus.cc:388] T 00000000000000000000000000000000 P face107a2b714ce888198f167e82a4b4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 10:21:08.045681 21803 raft_consensus.cc:482] T 00000000000000000000000000000000 P face107a2b714ce888198f167e82a4b4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 10:21:08.045764 21803 raft_consensus.cc:3037] T 00000000000000000000000000000000 P face107a2b714ce888198f167e82a4b4 [term 0 FOLLOWER]: Advancing to term 1
I20260619 10:21:08.046350 21803 raft_consensus.cc:504] T 00000000000000000000000000000000 P face107a2b714ce888198f167e82a4b4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "face107a2b714ce888198f167e82a4b4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36491 } }
I20260619 10:21:08.046444 21803 leader_election.cc:302] T 00000000000000000000000000000000 P face107a2b714ce888198f167e82a4b4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: face107a2b714ce888198f167e82a4b4; no voters:
I20260619 10:21:08.046592 21803 leader_election.cc:288] T 00000000000000000000000000000000 P face107a2b714ce888198f167e82a4b4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 10:21:08.046649 21808 raft_consensus.cc:2781] T 00000000000000000000000000000000 P face107a2b714ce888198f167e82a4b4 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 10:21:08.046777 21808 raft_consensus.cc:686] T 00000000000000000000000000000000 P face107a2b714ce888198f167e82a4b4 [term 1 LEADER]: Becoming Leader. State: Replica: face107a2b714ce888198f167e82a4b4, State: Running, Role: LEADER
I20260619 10:21:08.046808 21803 sys_catalog.cc:564] T 00000000000000000000000000000000 P face107a2b714ce888198f167e82a4b4 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 10:21:08.046897 21808 consensus_queue.cc:237] T 00000000000000000000000000000000 P face107a2b714ce888198f167e82a4b4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "face107a2b714ce888198f167e82a4b4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36491 } }
I20260619 10:21:08.047140 21808 sys_catalog.cc:455] T 00000000000000000000000000000000 P face107a2b714ce888198f167e82a4b4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader face107a2b714ce888198f167e82a4b4. Latest consensus state: current_term: 1 leader_uuid: "face107a2b714ce888198f167e82a4b4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "face107a2b714ce888198f167e82a4b4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36491 } } }
I20260619 10:21:08.047266 21808 sys_catalog.cc:458] T 00000000000000000000000000000000 P face107a2b714ce888198f167e82a4b4 [sys.catalog]: This master's current role is: LEADER
I20260619 10:21:08.047271 21810 sys_catalog.cc:455] T 00000000000000000000000000000000 P face107a2b714ce888198f167e82a4b4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "face107a2b714ce888198f167e82a4b4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "face107a2b714ce888198f167e82a4b4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36491 } } }
I20260619 10:21:08.047394 21810 sys_catalog.cc:458] T 00000000000000000000000000000000 P face107a2b714ce888198f167e82a4b4 [sys.catalog]: This master's current role is: LEADER
W20260619 10:21:08.048815 21823 catalog_manager.cc:1558] T 00000000000000000000000000000000 P face107a2b714ce888198f167e82a4b4: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 10:21:08.048914 21823 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 10:21:08.049019 21812 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 10:21:08.049240 21812 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 10:21:08.050383 21807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:21:08.043855 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.235:38183 (local address 127.0.0.1:36491)
0619 10:21:08.044025 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0619 10:21:08.044034 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 10:21:08.044064 (+ 30us) server_negotiation.cc:407] Connection header received
0619 10:21:08.045390 (+ 1326us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:21:08.045398 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:21:08.045459 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:21:08.045618 (+ 159us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:21:08.045837 (+ 219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:21:08.047061 (+ 1224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:21:08.048224 (+ 1163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:21:08.048828 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:21:08.049283 (+ 455us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:21:08.049710 (+ 427us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 10:21:08.049713 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 10:21:08.049714 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 10:21:08.049912 (+ 198us) server_negotiation.cc:922] calling sasl_server_start()
0619 10:21:08.049947 (+ 35us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 10:21:08.049960 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 10:21:08.050064 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0619 10:21:08.050172 (+ 108us) server_negotiation.cc:299] Negotiation successful
0619 10:21:08.050197 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":122,"thread_start_us":70,"threads_started":1}
I20260619 10:21:08.050796 21812 catalog_manager.cc:1347] Generated new cluster ID: c70d0d6f18ec4ca9a09097495f97fa3e
I20260619 10:21:08.050869 21812 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 10:21:08.051362 825 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 10:21:08.062026 21807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:21:08.058858 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.235:46201 (local address 127.0.0.1:36491)
0619 10:21:08.058891 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0619 10:21:08.058893 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 10:21:08.058951 (+ 58us) server_negotiation.cc:407] Connection header received
0619 10:21:08.059002 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:21:08.059003 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:21:08.059012 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:21:08.059070 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:21:08.059216 (+ 146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:21:08.059920 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:21:08.060838 (+ 918us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:21:08.061134 (+ 296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:21:08.061198 (+ 64us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:21:08.061797 (+ 599us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 10:21:08.061798 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 10:21:08.061798 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 10:21:08.061803 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0619 10:21:08.061816 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 10:21:08.061820 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 10:21:08.061880 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0619 10:21:08.061946 (+ 66us) server_negotiation.cc:299] Negotiation successful
0619 10:21:08.061955 (+ 9us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260619 10:21:08.067050 21812 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 10:21:08.067729 21812 catalog_manager.cc:1504] Loading token signing keys...
I20260619 10:21:08.071817 21812 catalog_manager.cc:5868] T 00000000000000000000000000000000 P face107a2b714ce888198f167e82a4b4: Generated new TSK 0
I20260619 10:21:08.071969 21812 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 10:21:08.077783 825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTU9OP6/build/release/bin/kudu with pid 21735
W20260619 10:21:08.081686 21720 connection.cc:535] client connection to 127.0.0.1:36491 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:36491): Connection reset by peer (error 104)
2026-06-19T10:21:08Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (1255 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781864224577058-825-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 10:21:08 dist-test-slave-6pt2 krb5kdc[21834](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 10:21:08 dist-test-slave-6pt2 krb5kdc[21834](info): set up 2 sockets
Jun 19 10:21:08 dist-test-slave-6pt2 krb5kdc[21834](info): commencing operation
krb5kdc: starting...
W20260619 10:21:09.090512 825 mini_kdc.cc:121] Time spent starting KDC: real 0.995s user 0.002s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781864224577058-825-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781864224577058-825-0/krb5kdc/test-user.keytab.
Jun 19 10:21:09 dist-test-slave-6pt2 krb5kdc[21834](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864469, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T10:21:09Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T10:21:09Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781864224577058-825-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781864224577058-825-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260619 10:21:09.199095 825 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTU9OP6/build/release/bin/kudu
/tmp/dist-test-taskTU9OP6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781864224577058-825-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781864224577058-825-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781864224577058-825-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781864224577058-825-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34921
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781864224577058-825-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:33463
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:34921
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781864224577058-825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781864224577058-825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781864224577058-825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 10:21:09.276355 21847 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 10:21:09.276525 21847 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 10:21:09.276600 21847 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 10:21:09.277743 21847 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 10:21:09.277783 21847 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 10:21:09.277798 21847 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 10:21:09.277812 21847 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 10:21:09.277823 21847 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 10:21:09.279199 21847 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:33463
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781864224577058-825-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781864224577058-825-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:34921
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781864224577058-825-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34921
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781864224577058-825-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21847
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781864224577058-825-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 09:32:44 UTC on ccycloud.quasar-gbtldp.root.comops.site
build id 4455440
I20260619 10:21:09.279357 21847 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 10:21:09.279510 21847 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 10:21:09.283946 21853 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 10:21:09.283941 21852 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 10:21:09.284019 21855 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 10:21:09.284071 21847 server_base.cc:1034] running on GCE node
I20260619 10:21:09.284364 21847 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 10:21:09.284685 21847 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 10:21:09.285840 21847 hybrid_clock.cc:648] HybridClock initialized: now 1781864469285835 us; error 34 us; skew 500 ppm
Jun 19 10:21:09 dist-test-slave-6pt2 krb5kdc[21834](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864469, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 10:21:09.292165 21847 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260619 10:21:09.292619 21847 webserver.cc:460] Webserver started at http://127.0.0.1:34689/ using document root <none> and password file <none>
I20260619 10:21:09.292835 21847 fs_manager.cc:359] Metadata directory not provided
I20260619 10:21:09.292899 21847 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 10:21:09.293051 21847 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 10:21:09.294019 21847 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781864224577058-825-0/minicluster-data/master-0/data/instance:
uuid: "a19bc07b4d4c4c5e85cc81ae3896d3f1"
format_stamp: "Formatted at 2026-06-19 10:21:09 on dist-test-slave-6pt2"
server_key: "5913fd7d432bca409bb7988a9df4364e"
server_key_iv: "83402473809258d108e9ba804dc3ee5e"
server_key_version: "encryptionkey@0"
I20260619 10:21:09.294353 21847 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781864224577058-825-0/minicluster-data/master-0/wal/instance:
uuid: "a19bc07b4d4c4c5e85cc81ae3896d3f1"
format_stamp: "Formatted at 2026-06-19 10:21:09 on dist-test-slave-6pt2"
server_key: "5913fd7d432bca409bb7988a9df4364e"
server_key_iv: "83402473809258d108e9ba804dc3ee5e"
server_key_version: "encryptionkey@0"
I20260619 10:21:09.295746 21847 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260619 10:21:09.296551 21862 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 10:21:09.296749 21847 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260619 10:21:09.296844 21847 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781864224577058-825-0/minicluster-data/master-0/data,/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781864224577058-825-0/minicluster-data/master-0/wal
uuid: "a19bc07b4d4c4c5e85cc81ae3896d3f1"
format_stamp: "Formatted at 2026-06-19 10:21:09 on dist-test-slave-6pt2"
server_key: "5913fd7d432bca409bb7988a9df4364e"
server_key_iv: "83402473809258d108e9ba804dc3ee5e"
server_key_version: "encryptionkey@0"
I20260619 10:21:09.296926 21847 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781864224577058-825-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781864224577058-825-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781864224577058-825-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 10:21:09.300136 21847 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 10:21:09.301939 21847 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 10:21:09.302096 21847 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 10:21:09.305976 21847 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:34921
I20260619 10:21:09.306012 21914 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:34921 every 8 connection(s)
I20260619 10:21:09.306313 21847 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781864224577058-825-0/minicluster-data/master-0/data/info.pb
I20260619 10:21:09.307233 21915 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 10:21:09.309252 21915 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a19bc07b4d4c4c5e85cc81ae3896d3f1: Bootstrap starting.
I20260619 10:21:09.309815 21915 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a19bc07b4d4c4c5e85cc81ae3896d3f1: Neither blocks nor log segments found. Creating new log.
I20260619 10:21:09.310088 21915 log.cc:824] T 00000000000000000000000000000000 P a19bc07b4d4c4c5e85cc81ae3896d3f1: Log is configured to *not* fsync() on all Append() calls
I20260619 10:21:09.310804 21915 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a19bc07b4d4c4c5e85cc81ae3896d3f1: No bootstrap required, opened a new log
I20260619 10:21:09.312278 21915 raft_consensus.cc:348] T 00000000000000000000000000000000 P a19bc07b4d4c4c5e85cc81ae3896d3f1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a19bc07b4d4c4c5e85cc81ae3896d3f1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34921 } }
I20260619 10:21:09.312397 21915 raft_consensus.cc:374] T 00000000000000000000000000000000 P a19bc07b4d4c4c5e85cc81ae3896d3f1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 10:21:09.312438 21915 raft_consensus.cc:729] T 00000000000000000000000000000000 P a19bc07b4d4c4c5e85cc81ae3896d3f1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a19bc07b4d4c4c5e85cc81ae3896d3f1, State: Initialized, Role: FOLLOWER
I20260619 10:21:09.312628 21915 consensus_queue.cc:260] T 00000000000000000000000000000000 P a19bc07b4d4c4c5e85cc81ae3896d3f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a19bc07b4d4c4c5e85cc81ae3896d3f1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34921 } }
I20260619 10:21:09.312687 21915 raft_consensus.cc:388] T 00000000000000000000000000000000 P a19bc07b4d4c4c5e85cc81ae3896d3f1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 10:21:09.312723 21915 raft_consensus.cc:482] T 00000000000000000000000000000000 P a19bc07b4d4c4c5e85cc81ae3896d3f1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 10:21:09.312788 21915 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a19bc07b4d4c4c5e85cc81ae3896d3f1 [term 0 FOLLOWER]: Advancing to term 1
I20260619 10:21:09.313028 825 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTU9OP6/build/release/bin/kudu as pid 21847
I20260619 10:21:09.313140 825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781864224577058-825-0/minicluster-data/master-0/wal/instance
I20260619 10:21:09.313299 825 external_mini_cluster.cc:1428] Setting key 7339d7576901e06ab19db2a0b7de1c64
I20260619 10:21:09.313460 21915 raft_consensus.cc:504] T 00000000000000000000000000000000 P a19bc07b4d4c4c5e85cc81ae3896d3f1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a19bc07b4d4c4c5e85cc81ae3896d3f1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34921 } }
I20260619 10:21:09.313566 21915 leader_election.cc:302] T 00000000000000000000000000000000 P a19bc07b4d4c4c5e85cc81ae3896d3f1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a19bc07b4d4c4c5e85cc81ae3896d3f1; no voters:
I20260619 10:21:09.313732 21915 leader_election.cc:288] T 00000000000000000000000000000000 P a19bc07b4d4c4c5e85cc81ae3896d3f1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 10:21:09.313778 21918 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a19bc07b4d4c4c5e85cc81ae3896d3f1 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 10:21:09.313974 21918 raft_consensus.cc:686] T 00000000000000000000000000000000 P a19bc07b4d4c4c5e85cc81ae3896d3f1 [term 1 LEADER]: Becoming Leader. State: Replica: a19bc07b4d4c4c5e85cc81ae3896d3f1, State: Running, Role: LEADER
I20260619 10:21:09.314023 21915 sys_catalog.cc:564] T 00000000000000000000000000000000 P a19bc07b4d4c4c5e85cc81ae3896d3f1 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 10:21:09.314121 21918 consensus_queue.cc:237] T 00000000000000000000000000000000 P a19bc07b4d4c4c5e85cc81ae3896d3f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a19bc07b4d4c4c5e85cc81ae3896d3f1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34921 } }
I20260619 10:21:09.314587 21921 sys_catalog.cc:455] T 00000000000000000000000000000000 P a19bc07b4d4c4c5e85cc81ae3896d3f1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a19bc07b4d4c4c5e85cc81ae3896d3f1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a19bc07b4d4c4c5e85cc81ae3896d3f1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34921 } } }
I20260619 10:21:09.314642 21922 sys_catalog.cc:455] T 00000000000000000000000000000000 P a19bc07b4d4c4c5e85cc81ae3896d3f1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a19bc07b4d4c4c5e85cc81ae3896d3f1. Latest consensus state: current_term: 1 leader_uuid: "a19bc07b4d4c4c5e85cc81ae3896d3f1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a19bc07b4d4c4c5e85cc81ae3896d3f1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34921 } } }
I20260619 10:21:09.314777 21921 sys_catalog.cc:458] T 00000000000000000000000000000000 P a19bc07b4d4c4c5e85cc81ae3896d3f1 [sys.catalog]: This master's current role is: LEADER
I20260619 10:21:09.314790 21922 sys_catalog.cc:458] T 00000000000000000000000000000000 P a19bc07b4d4c4c5e85cc81ae3896d3f1 [sys.catalog]: This master's current role is: LEADER
W20260619 10:21:09.315869 21936 catalog_manager.cc:1558] T 00000000000000000000000000000000 P a19bc07b4d4c4c5e85cc81ae3896d3f1: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 10:21:09.315924 21936 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 10:21:09.316012 21929 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 10:21:09.316224 21929 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 10:21:09.317210 21920 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:21:09.313730 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.235:33863 (local address 127.0.0.1:34921)
0619 10:21:09.313891 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0619 10:21:09.313900 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 10:21:09.315690 (+ 1790us) server_negotiation.cc:407] Connection header received
0619 10:21:09.315863 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:21:09.315870 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:21:09.315918 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:21:09.316078 (+ 160us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:21:09.316756 (+ 678us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 10:21:09.316759 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 10:21:09.316760 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 10:21:09.316764 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0619 10:21:09.316806 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 10:21:09.316821 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 10:21:09.316890 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0619 10:21:09.316963 (+ 73us) server_negotiation.cc:299] Negotiation successful
0619 10:21:09.317005 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":108,"thread_start_us":77,"threads_started":1}
I20260619 10:21:09.317745 21929 catalog_manager.cc:1347] Generated new cluster ID: fc12724e6a9d482d9d3b47837aeeb2a9
I20260619 10:21:09.317790 21929 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 10:21:09.318115 825 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 10:21:09.320425 21920 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:21:09.319474 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.235:37645 (local address 127.0.0.1:34921)
0619 10:21:09.319503 (+ 29us) server_negotiation.cc:206] Beginning negotiation
0619 10:21:09.319505 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 10:21:09.319616 (+ 111us) server_negotiation.cc:407] Connection header received
0619 10:21:09.319669 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:21:09.319670 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:21:09.319673 (+ 3us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:21:09.319719 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:21:09.320234 (+ 515us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 10:21:09.320235 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 10:21:09.320235 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 10:21:09.320239 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0619 10:21:09.320250 (+ 11us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 10:21:09.320254 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 10:21:09.320302 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0619 10:21:09.320354 (+ 52us) server_negotiation.cc:299] Negotiation successful
0619 10:21:09.320364 (+ 10us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260619 10:21:09.343286 21929 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 10:21:09.343433 21929 catalog_manager.cc:1504] Loading token signing keys...
I20260619 10:21:09.345978 21929 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a19bc07b4d4c4c5e85cc81ae3896d3f1: Generated new TSK 0
I20260619 10:21:09.346127 21929 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 10:21:09.348315 825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTU9OP6/build/release/bin/kudu with pid 21847
2026-06-19T10:21:09Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (1270 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781864224577058-825-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 10:21:09 dist-test-slave-6pt2 krb5kdc[21946](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 10:21:09 dist-test-slave-6pt2 krb5kdc[21946](info): set up 2 sockets
Jun 19 10:21:09 dist-test-slave-6pt2 krb5kdc[21946](info): commencing operation
krb5kdc: starting...
W20260619 10:21:10.350459 825 mini_kdc.cc:121] Time spent starting KDC: real 0.985s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781864224577058-825-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781864224577058-825-0/krb5kdc/test-user.keytab.
Jun 19 10:21:10 dist-test-slave-6pt2 krb5kdc[21946](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864470, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T10:21:10Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T10:21:10Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781864224577058-825-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781864224577058-825-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260619 10:21:10.458261 825 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTU9OP6/build/release/bin/kudu
/tmp/dist-test-taskTU9OP6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781864224577058-825-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781864224577058-825-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781864224577058-825-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781864224577058-825-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33581
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781864224577058-825-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:43613
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:33581
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781864224577058-825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781864224577058-825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781864224577058-825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 10:21:10.533737 21959 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 10:21:10.533911 21959 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 10:21:10.533931 21959 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 10:21:10.535174 21959 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 10:21:10.535233 21959 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 10:21:10.535248 21959 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 10:21:10.535296 21959 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 10:21:10.535315 21959 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 10:21:10.536731 21959 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:43613
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781864224577058-825-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781864224577058-825-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:33581
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781864224577058-825-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33581
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781864224577058-825-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21959
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781864224577058-825-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 09:32:44 UTC on ccycloud.quasar-gbtldp.root.comops.site
build id 4455440
I20260619 10:21:10.536957 21959 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 10:21:10.537222 21959 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260619 10:21:10.541848 21959 server_base.cc:1034] running on GCE node
W20260619 10:21:10.541814 21965 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 10:21:10.541821 21964 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 10:21:10.541831 21967 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 10:21:10.542232 21959 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 10:21:10.542527 21959 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 10:21:10.543685 21959 hybrid_clock.cc:648] HybridClock initialized: now 1781864470543696 us; error 48 us; skew 500 ppm
Jun 19 10:21:10 dist-test-slave-6pt2 krb5kdc[21946](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864470, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 10:21:10.545940 21959 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260619 10:21:10.546368 21959 webserver.cc:460] Webserver started at http://127.0.0.1:36785/ using document root <none> and password file <none>
I20260619 10:21:10.546581 21959 fs_manager.cc:359] Metadata directory not provided
I20260619 10:21:10.546671 21959 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 10:21:10.546798 21959 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 10:21:10.547757 21959 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781864224577058-825-0/minicluster-data/master-0/data/instance:
uuid: "3b8dad50c48a426ebf9a797ac508fe60"
format_stamp: "Formatted at 2026-06-19 10:21:10 on dist-test-slave-6pt2"
server_key: "80d8e1ebb7014f17922b1b8b3e0edd12"
server_key_iv: "93f5f59e610dc0524bd033365567337e"
server_key_version: "encryptionkey@0"
I20260619 10:21:10.548099 21959 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781864224577058-825-0/minicluster-data/master-0/wal/instance:
uuid: "3b8dad50c48a426ebf9a797ac508fe60"
format_stamp: "Formatted at 2026-06-19 10:21:10 on dist-test-slave-6pt2"
server_key: "80d8e1ebb7014f17922b1b8b3e0edd12"
server_key_iv: "93f5f59e610dc0524bd033365567337e"
server_key_version: "encryptionkey@0"
I20260619 10:21:10.549368 21959 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 10:21:10.550134 21974 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 10:21:10.550330 21959 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 10:21:10.550421 21959 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781864224577058-825-0/minicluster-data/master-0/data,/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781864224577058-825-0/minicluster-data/master-0/wal
uuid: "3b8dad50c48a426ebf9a797ac508fe60"
format_stamp: "Formatted at 2026-06-19 10:21:10 on dist-test-slave-6pt2"
server_key: "80d8e1ebb7014f17922b1b8b3e0edd12"
server_key_iv: "93f5f59e610dc0524bd033365567337e"
server_key_version: "encryptionkey@0"
I20260619 10:21:10.550490 21959 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781864224577058-825-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781864224577058-825-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781864224577058-825-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 10:21:10.574774 21959 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 10:21:10.576516 21959 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 10:21:10.576666 21959 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 10:21:10.580303 21959 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:33581
I20260619 10:21:10.580339 22026 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:33581 every 8 connection(s)
I20260619 10:21:10.580681 21959 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781864224577058-825-0/minicluster-data/master-0/data/info.pb
I20260619 10:21:10.581501 22027 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 10:21:10.582475 825 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTU9OP6/build/release/bin/kudu as pid 21959
I20260619 10:21:10.582569 825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781864224577058-825-0/minicluster-data/master-0/wal/instance
I20260619 10:21:10.582793 825 external_mini_cluster.cc:1428] Setting key aaf2cbc19d2b653db80131a11424f738
I20260619 10:21:10.583981 22027 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3b8dad50c48a426ebf9a797ac508fe60: Bootstrap starting.
I20260619 10:21:10.584524 22027 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3b8dad50c48a426ebf9a797ac508fe60: Neither blocks nor log segments found. Creating new log.
I20260619 10:21:10.584829 22027 log.cc:824] T 00000000000000000000000000000000 P 3b8dad50c48a426ebf9a797ac508fe60: Log is configured to *not* fsync() on all Append() calls
I20260619 10:21:10.585482 22027 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3b8dad50c48a426ebf9a797ac508fe60: No bootstrap required, opened a new log
I20260619 10:21:10.587262 22027 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3b8dad50c48a426ebf9a797ac508fe60 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b8dad50c48a426ebf9a797ac508fe60" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33581 } }
I20260619 10:21:10.587427 22027 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3b8dad50c48a426ebf9a797ac508fe60 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 10:21:10.587491 22027 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3b8dad50c48a426ebf9a797ac508fe60 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3b8dad50c48a426ebf9a797ac508fe60, State: Initialized, Role: FOLLOWER
I20260619 10:21:10.587635 22027 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3b8dad50c48a426ebf9a797ac508fe60 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b8dad50c48a426ebf9a797ac508fe60" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33581 } }
I20260619 10:21:10.587716 22027 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3b8dad50c48a426ebf9a797ac508fe60 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 10:21:10.587757 22027 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3b8dad50c48a426ebf9a797ac508fe60 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 10:21:10.587805 22027 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3b8dad50c48a426ebf9a797ac508fe60 [term 0 FOLLOWER]: Advancing to term 1
I20260619 10:21:10.588303 22027 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3b8dad50c48a426ebf9a797ac508fe60 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b8dad50c48a426ebf9a797ac508fe60" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33581 } }
I20260619 10:21:10.588414 22027 leader_election.cc:302] T 00000000000000000000000000000000 P 3b8dad50c48a426ebf9a797ac508fe60 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3b8dad50c48a426ebf9a797ac508fe60; no voters:
I20260619 10:21:10.588601 22027 leader_election.cc:288] T 00000000000000000000000000000000 P 3b8dad50c48a426ebf9a797ac508fe60 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 10:21:10.588675 22032 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3b8dad50c48a426ebf9a797ac508fe60 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 10:21:10.588837 22027 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3b8dad50c48a426ebf9a797ac508fe60 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 10:21:10.588842 22032 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3b8dad50c48a426ebf9a797ac508fe60 [term 1 LEADER]: Becoming Leader. State: Replica: 3b8dad50c48a426ebf9a797ac508fe60, State: Running, Role: LEADER
I20260619 10:21:10.588997 22032 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3b8dad50c48a426ebf9a797ac508fe60 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b8dad50c48a426ebf9a797ac508fe60" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33581 } }
Jun 19 10:21:10 dist-test-slave-6pt2 krb5kdc[21946](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864470, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260619 10:21:10.589319 22032 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3b8dad50c48a426ebf9a797ac508fe60 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3b8dad50c48a426ebf9a797ac508fe60. Latest consensus state: current_term: 1 leader_uuid: "3b8dad50c48a426ebf9a797ac508fe60" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b8dad50c48a426ebf9a797ac508fe60" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33581 } } }
I20260619 10:21:10.589392 22032 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3b8dad50c48a426ebf9a797ac508fe60 [sys.catalog]: This master's current role is: LEADER
I20260619 10:21:10.589552 22033 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3b8dad50c48a426ebf9a797ac508fe60 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3b8dad50c48a426ebf9a797ac508fe60" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b8dad50c48a426ebf9a797ac508fe60" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33581 } } }
I20260619 10:21:10.589607 22033 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3b8dad50c48a426ebf9a797ac508fe60 [sys.catalog]: This master's current role is: LEADER
I20260619 10:21:10.589936 22041 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 10:21:10.590370 22041 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 10:21:10.592180 22041 catalog_manager.cc:1347] Generated new cluster ID: 19fe9d79152a4b26ad6b8703079423c7
I20260619 10:21:10.592231 22041 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 10:21:10.593320 22030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:21:10.583275 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.235:54301 (local address 127.0.0.1:33581)
0619 10:21:10.583464 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0619 10:21:10.583473 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 10:21:10.583504 (+ 31us) server_negotiation.cc:407] Connection header received
0619 10:21:10.583895 (+ 391us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:21:10.583901 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:21:10.583948 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:21:10.584102 (+ 154us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:21:10.584738 (+ 636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:21:10.585671 (+ 933us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:21:10.586429 (+ 758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:21:10.586862 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:21:10.586975 (+ 113us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:21:10.589721 (+ 2746us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 10:21:10.589726 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 10:21:10.589726 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 10:21:10.589738 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0619 10:21:10.591677 (+ 1939us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:21:10.592137 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:21:10.592139 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:21:10.592140 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:21:10.592247 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:21:10.592495 (+ 248us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:21:10.592496 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:21:10.592497 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:21:10.592723 (+ 226us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 10:21:10.592892 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0619 10:21:10.593028 (+ 136us) server_negotiation.cc:299] Negotiation successful
0619 10:21:10.593129 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":142,"thread_start_us":76,"threads_started":1}
I20260619 10:21:10.599267 825 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 10:21:10.609701 22030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:21:10.600585 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.235:52423 (local address 127.0.0.1:33581)
0619 10:21:10.600619 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0619 10:21:10.600621 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 10:21:10.602794 (+ 2173us) server_negotiation.cc:407] Connection header received
0619 10:21:10.602862 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:21:10.602863 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:21:10.602873 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:21:10.602933 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:21:10.603494 (+ 561us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:21:10.603962 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:21:10.604672 (+ 710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:21:10.604998 (+ 326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:21:10.605065 (+ 67us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:21:10.606629 (+ 1564us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 10:21:10.606631 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 10:21:10.606631 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 10:21:10.606638 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 10:21:10.608334 (+ 1696us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:21:10.608777 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:21:10.608778 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:21:10.608778 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:21:10.608831 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:21:10.609080 (+ 249us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:21:10.609081 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:21:10.609081 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:21:10.609203 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 10:21:10.609281 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0619 10:21:10.609523 (+ 242us) server_negotiation.cc:299] Negotiation successful
0619 10:21:10.609589 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260619 10:21:10.610738 22041 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 10:21:10.611480 22041 catalog_manager.cc:1504] Loading token signing keys...
I20260619 10:21:10.614363 22041 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3b8dad50c48a426ebf9a797ac508fe60: Generated new TSK 0
I20260619 10:21:10.614517 22041 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 10:21:10.618243 825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTU9OP6/build/release/bin/kudu with pid 21959
W20260619 10:21:10.622130 21944 connection.cc:535] client connection to 127.0.0.1:33581 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:33581): Connection reset by peer (error 104)
2026-06-19T10:21:10Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (1269 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781864224577058-825-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 10:21:10 dist-test-slave-6pt2 krb5kdc[22058](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 10:21:10 dist-test-slave-6pt2 krb5kdc[22058](info): set up 2 sockets
Jun 19 10:21:10 dist-test-slave-6pt2 krb5kdc[22058](info): commencing operation
krb5kdc: starting...
W20260619 10:21:11.634476 825 mini_kdc.cc:121] Time spent starting KDC: real 0.999s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781864224577058-825-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781864224577058-825-0/krb5kdc/test-user.keytab.
Jun 19 10:21:11 dist-test-slave-6pt2 krb5kdc[22058](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864471, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T10:21:11Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T10:21:11Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781864224577058-825-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781864224577058-825-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260619 10:21:11.739265 825 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTU9OP6/build/release/bin/kudu
/tmp/dist-test-taskTU9OP6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781864224577058-825-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781864224577058-825-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781864224577058-825-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781864224577058-825-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41209
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781864224577058-825-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36827
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:41209
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781864224577058-825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781864224577058-825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781864224577058-825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 10:21:11.815766 22071 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 10:21:11.815958 22071 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 10:21:11.815986 22071 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 10:21:11.817019 22071 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 10:21:11.817050 22071 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 10:21:11.817065 22071 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 10:21:11.817076 22071 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 10:21:11.817087 22071 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 10:21:11.818447 22071 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36827
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781864224577058-825-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781864224577058-825-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:41209
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781864224577058-825-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41209
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781864224577058-825-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22071
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781864224577058-825-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 09:32:44 UTC on ccycloud.quasar-gbtldp.root.comops.site
build id 4455440
I20260619 10:21:11.818717 22071 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 10:21:11.818890 22071 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260619 10:21:11.823406 22071 server_base.cc:1034] running on GCE node
W20260619 10:21:11.823392 22076 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 10:21:11.823392 22079 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 10:21:11.823402 22077 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 10:21:11.823799 22071 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 10:21:11.824079 22071 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 10:21:11.825240 22071 hybrid_clock.cc:648] HybridClock initialized: now 1781864471825234 us; error 34 us; skew 500 ppm
Jun 19 10:21:11 dist-test-slave-6pt2 krb5kdc[22058](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864471, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 10:21:11.827554 22071 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260619 10:21:11.827960 22071 webserver.cc:460] Webserver started at http://127.0.0.1:39831/ using document root <none> and password file <none>
I20260619 10:21:11.828137 22071 fs_manager.cc:359] Metadata directory not provided
I20260619 10:21:11.828182 22071 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 10:21:11.828263 22071 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 10:21:11.829172 22071 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781864224577058-825-0/minicluster-data/master-0/data/instance:
uuid: "9527bb3cfcd04bb09440355a03bc4d64"
format_stamp: "Formatted at 2026-06-19 10:21:11 on dist-test-slave-6pt2"
server_key: "f6559fe1a30ed6c9737c3cd5de698884"
server_key_iv: "9fbb0f9f1287b425c4feae29266e5a5e"
server_key_version: "encryptionkey@0"
I20260619 10:21:11.829455 22071 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781864224577058-825-0/minicluster-data/master-0/wal/instance:
uuid: "9527bb3cfcd04bb09440355a03bc4d64"
format_stamp: "Formatted at 2026-06-19 10:21:11 on dist-test-slave-6pt2"
server_key: "f6559fe1a30ed6c9737c3cd5de698884"
server_key_iv: "9fbb0f9f1287b425c4feae29266e5a5e"
server_key_version: "encryptionkey@0"
I20260619 10:21:11.830756 22071 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 10:21:11.831426 22086 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 10:21:11.831630 22071 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 10:21:11.831727 22071 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781864224577058-825-0/minicluster-data/master-0/data,/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781864224577058-825-0/minicluster-data/master-0/wal
uuid: "9527bb3cfcd04bb09440355a03bc4d64"
format_stamp: "Formatted at 2026-06-19 10:21:11 on dist-test-slave-6pt2"
server_key: "f6559fe1a30ed6c9737c3cd5de698884"
server_key_iv: "9fbb0f9f1287b425c4feae29266e5a5e"
server_key_version: "encryptionkey@0"
I20260619 10:21:11.831815 22071 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781864224577058-825-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781864224577058-825-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781864224577058-825-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 10:21:11.843626 22071 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 10:21:11.845286 22071 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 10:21:11.845436 22071 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 10:21:11.849090 22071 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:41209
I20260619 10:21:11.849119 22138 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:41209 every 8 connection(s)
I20260619 10:21:11.849432 22071 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781864224577058-825-0/minicluster-data/master-0/data/info.pb
I20260619 10:21:11.850507 22139 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 10:21:11.852754 22139 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9527bb3cfcd04bb09440355a03bc4d64: Bootstrap starting.
I20260619 10:21:11.853196 825 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTU9OP6/build/release/bin/kudu as pid 22071
I20260619 10:21:11.853261 22139 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9527bb3cfcd04bb09440355a03bc4d64: Neither blocks nor log segments found. Creating new log.
I20260619 10:21:11.853329 825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781864224577058-825-0/minicluster-data/master-0/wal/instance
I20260619 10:21:11.853514 825 external_mini_cluster.cc:1428] Setting key dc7fb5cb8924fce3595616fff443a2ae
I20260619 10:21:11.853565 22139 log.cc:824] T 00000000000000000000000000000000 P 9527bb3cfcd04bb09440355a03bc4d64: Log is configured to *not* fsync() on all Append() calls
I20260619 10:21:11.854333 22139 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9527bb3cfcd04bb09440355a03bc4d64: No bootstrap required, opened a new log
I20260619 10:21:11.856949 22139 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9527bb3cfcd04bb09440355a03bc4d64 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9527bb3cfcd04bb09440355a03bc4d64" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41209 } }
I20260619 10:21:11.857079 22139 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9527bb3cfcd04bb09440355a03bc4d64 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 10:21:11.857101 22139 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9527bb3cfcd04bb09440355a03bc4d64 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9527bb3cfcd04bb09440355a03bc4d64, State: Initialized, Role: FOLLOWER
I20260619 10:21:11.857224 22139 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9527bb3cfcd04bb09440355a03bc4d64 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9527bb3cfcd04bb09440355a03bc4d64" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41209 } }
I20260619 10:21:11.857295 22139 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9527bb3cfcd04bb09440355a03bc4d64 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 10:21:11.857316 22139 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9527bb3cfcd04bb09440355a03bc4d64 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 10:21:11.857393 22139 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9527bb3cfcd04bb09440355a03bc4d64 [term 0 FOLLOWER]: Advancing to term 1
I20260619 10:21:11.858026 22139 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9527bb3cfcd04bb09440355a03bc4d64 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9527bb3cfcd04bb09440355a03bc4d64" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41209 } }
I20260619 10:21:11.858127 22139 leader_election.cc:302] T 00000000000000000000000000000000 P 9527bb3cfcd04bb09440355a03bc4d64 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9527bb3cfcd04bb09440355a03bc4d64; no voters:
I20260619 10:21:11.858274 22139 leader_election.cc:288] T 00000000000000000000000000000000 P 9527bb3cfcd04bb09440355a03bc4d64 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 10:21:11.858340 22144 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9527bb3cfcd04bb09440355a03bc4d64 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 10:21:11.858479 22144 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9527bb3cfcd04bb09440355a03bc4d64 [term 1 LEADER]: Becoming Leader. State: Replica: 9527bb3cfcd04bb09440355a03bc4d64, State: Running, Role: LEADER
I20260619 10:21:11.858505 22139 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9527bb3cfcd04bb09440355a03bc4d64 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 10:21:11.858561 22144 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9527bb3cfcd04bb09440355a03bc4d64 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9527bb3cfcd04bb09440355a03bc4d64" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41209 } }
I20260619 10:21:11.858873 22144 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9527bb3cfcd04bb09440355a03bc4d64 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9527bb3cfcd04bb09440355a03bc4d64. Latest consensus state: current_term: 1 leader_uuid: "9527bb3cfcd04bb09440355a03bc4d64" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9527bb3cfcd04bb09440355a03bc4d64" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41209 } } }
I20260619 10:21:11.858958 22144 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9527bb3cfcd04bb09440355a03bc4d64 [sys.catalog]: This master's current role is: LEADER
I20260619 10:21:11.859167 22145 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9527bb3cfcd04bb09440355a03bc4d64 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9527bb3cfcd04bb09440355a03bc4d64" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9527bb3cfcd04bb09440355a03bc4d64" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41209 } } }
I20260619 10:21:11.859334 22145 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9527bb3cfcd04bb09440355a03bc4d64 [sys.catalog]: This master's current role is: LEADER
I20260619 10:21:11.859182 22150 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 10:21:11.859855 22150 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 10:21:11.860139 22143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:21:11.853994 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.235:42871 (local address 127.0.0.1:41209)
0619 10:21:11.854239 (+ 245us) server_negotiation.cc:206] Beginning negotiation
0619 10:21:11.854247 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 10:21:11.854278 (+ 31us) server_negotiation.cc:407] Connection header received
0619 10:21:11.854848 (+ 570us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:21:11.854856 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:21:11.854918 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:21:11.855061 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:21:11.855229 (+ 168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:21:11.856605 (+ 1376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:21:11.857745 (+ 1140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:21:11.858365 (+ 620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:21:11.858828 (+ 463us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:21:11.859265 (+ 437us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 10:21:11.859270 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 10:21:11.859270 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 10:21:11.859461 (+ 191us) server_negotiation.cc:922] calling sasl_server_start()
0619 10:21:11.859511 (+ 50us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 10:21:11.859527 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 10:21:11.859616 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0619 10:21:11.859877 (+ 261us) server_negotiation.cc:299] Negotiation successful
0619 10:21:11.859905 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":177,"thread_start_us":121,"threads_started":1}
I20260619 10:21:11.861110 825 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 10:21:11.861526 22150 catalog_manager.cc:1347] Generated new cluster ID: 9fdfcb45c49e43d88356c40327b99ad3
I20260619 10:21:11.861572 22150 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 10:21:11.869537 22143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:21:11.862559 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.235:44835 (local address 127.0.0.1:41209)
0619 10:21:11.862595 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0619 10:21:11.862615 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0619 10:21:11.866847 (+ 4232us) server_negotiation.cc:407] Connection header received
0619 10:21:11.866911 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:21:11.866912 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:21:11.866922 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:21:11.866977 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:21:11.867140 (+ 163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:21:11.867610 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:21:11.868337 (+ 727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:21:11.868572 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:21:11.868642 (+ 70us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:21:11.869297 (+ 655us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 10:21:11.869298 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 10:21:11.869298 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 10:21:11.869302 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0619 10:21:11.869315 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 10:21:11.869319 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 10:21:11.869370 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0619 10:21:11.869427 (+ 57us) server_negotiation.cc:299] Negotiation successful
0619 10:21:11.869436 (+ 9us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260619 10:21:11.872242 22150 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 10:21:11.872720 22150 catalog_manager.cc:1504] Loading token signing keys...
I20260619 10:21:11.885852 22150 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9527bb3cfcd04bb09440355a03bc4d64: Generated new TSK 0
I20260619 10:21:11.886027 22150 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 10:21:11.897181 825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTU9OP6/build/release/bin/kudu with pid 22071
W20260619 10:21:11.901283 22056 connection.cc:535] client connection to 127.0.0.1:41209 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:41209): Connection reset by peer (error 104)
2026-06-19T10:21:11Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (1279 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781864224577058-825-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 10:21:11 dist-test-slave-6pt2 krb5kdc[22170](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 10:21:11 dist-test-slave-6pt2 krb5kdc[22170](info): set up 2 sockets
Jun 19 10:21:11 dist-test-slave-6pt2 krb5kdc[22170](info): commencing operation
krb5kdc: starting...
W20260619 10:21:12.914835 825 mini_kdc.cc:121] Time spent starting KDC: real 1.000s user 0.004s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781864224577058-825-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781864224577058-825-0/krb5kdc/test-user.keytab.
Jun 19 10:21:12 dist-test-slave-6pt2 krb5kdc[22170](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864472, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T10:21:12Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T10:21:12Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781864224577058-825-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781864224577058-825-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260619 10:21:13.024689 825 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTU9OP6/build/release/bin/kudu
/tmp/dist-test-taskTU9OP6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781864224577058-825-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781864224577058-825-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781864224577058-825-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781864224577058-825-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42645
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781864224577058-825-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:44763
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42645
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781864224577058-825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781864224577058-825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781864224577058-825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 10:21:13.102969 22183 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 10:21:13.103204 22183 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 10:21:13.103237 22183 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 10:21:13.104417 22183 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 10:21:13.104483 22183 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 10:21:13.104511 22183 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 10:21:13.104534 22183 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 10:21:13.104559 22183 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 10:21:13.106349 22183 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:44763
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781864224577058-825-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781864224577058-825-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42645
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781864224577058-825-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42645
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781864224577058-825-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22183
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781864224577058-825-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 09:32:44 UTC on ccycloud.quasar-gbtldp.root.comops.site
build id 4455440
I20260619 10:21:13.106578 22183 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 10:21:13.106832 22183 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 10:21:13.111352 22188 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 10:21:13.111352 22191 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 10:21:13.111348 22189 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 10:21:13.111452 22183 server_base.cc:1034] running on GCE node
I20260619 10:21:13.111799 22183 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 10:21:13.112102 22183 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 10:21:13.113267 22183 hybrid_clock.cc:648] HybridClock initialized: now 1781864473113245 us; error 33 us; skew 500 ppm
Jun 19 10:21:13 dist-test-slave-6pt2 krb5kdc[22170](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864473, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 10:21:13.115793 22183 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260619 10:21:13.116273 22183 webserver.cc:460] Webserver started at http://127.0.0.1:42075/ using document root <none> and password file <none>
I20260619 10:21:13.116483 22183 fs_manager.cc:359] Metadata directory not provided
I20260619 10:21:13.116523 22183 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 10:21:13.116609 22183 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 10:21:13.117616 22183 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781864224577058-825-0/minicluster-data/master-0/data/instance:
uuid: "fbacc683344144319504f35018d9dc1c"
format_stamp: "Formatted at 2026-06-19 10:21:13 on dist-test-slave-6pt2"
server_key: "8276d52e7bccdc74d725dd165f0b6a8b"
server_key_iv: "a79f0b4b5c6f3ca5e3460f4bc64a6a99"
server_key_version: "encryptionkey@0"
I20260619 10:21:13.117905 22183 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781864224577058-825-0/minicluster-data/master-0/wal/instance:
uuid: "fbacc683344144319504f35018d9dc1c"
format_stamp: "Formatted at 2026-06-19 10:21:13 on dist-test-slave-6pt2"
server_key: "8276d52e7bccdc74d725dd165f0b6a8b"
server_key_iv: "a79f0b4b5c6f3ca5e3460f4bc64a6a99"
server_key_version: "encryptionkey@0"
I20260619 10:21:13.119249 22183 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 10:21:13.119947 22198 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 10:21:13.120115 22183 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 10:21:13.120185 22183 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781864224577058-825-0/minicluster-data/master-0/data,/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781864224577058-825-0/minicluster-data/master-0/wal
uuid: "fbacc683344144319504f35018d9dc1c"
format_stamp: "Formatted at 2026-06-19 10:21:13 on dist-test-slave-6pt2"
server_key: "8276d52e7bccdc74d725dd165f0b6a8b"
server_key_iv: "a79f0b4b5c6f3ca5e3460f4bc64a6a99"
server_key_version: "encryptionkey@0"
I20260619 10:21:13.120237 22183 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781864224577058-825-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781864224577058-825-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781864224577058-825-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 10:21:13.123540 22183 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 10:21:13.125312 22183 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 10:21:13.125461 22183 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 10:21:13.129264 22183 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42645
I20260619 10:21:13.129292 22250 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42645 every 8 connection(s)
I20260619 10:21:13.129592 22183 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781864224577058-825-0/minicluster-data/master-0/data/info.pb
I20260619 10:21:13.130435 22251 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 10:21:13.132525 22251 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fbacc683344144319504f35018d9dc1c: Bootstrap starting.
I20260619 10:21:13.133014 22251 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fbacc683344144319504f35018d9dc1c: Neither blocks nor log segments found. Creating new log.
I20260619 10:21:13.133232 22251 log.cc:824] T 00000000000000000000000000000000 P fbacc683344144319504f35018d9dc1c: Log is configured to *not* fsync() on all Append() calls
I20260619 10:21:13.133797 22251 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fbacc683344144319504f35018d9dc1c: No bootstrap required, opened a new log
I20260619 10:21:13.135456 22251 raft_consensus.cc:348] T 00000000000000000000000000000000 P fbacc683344144319504f35018d9dc1c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fbacc683344144319504f35018d9dc1c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42645 } }
I20260619 10:21:13.135596 22251 raft_consensus.cc:374] T 00000000000000000000000000000000 P fbacc683344144319504f35018d9dc1c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 10:21:13.135617 22251 raft_consensus.cc:729] T 00000000000000000000000000000000 P fbacc683344144319504f35018d9dc1c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbacc683344144319504f35018d9dc1c, State: Initialized, Role: FOLLOWER
I20260619 10:21:13.135713 22251 consensus_queue.cc:260] T 00000000000000000000000000000000 P fbacc683344144319504f35018d9dc1c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fbacc683344144319504f35018d9dc1c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42645 } }
I20260619 10:21:13.135792 22251 raft_consensus.cc:388] T 00000000000000000000000000000000 P fbacc683344144319504f35018d9dc1c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 10:21:13.135836 22251 raft_consensus.cc:482] T 00000000000000000000000000000000 P fbacc683344144319504f35018d9dc1c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 10:21:13.135885 22251 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fbacc683344144319504f35018d9dc1c [term 0 FOLLOWER]: Advancing to term 1
I20260619 10:21:13.136418 22251 raft_consensus.cc:504] T 00000000000000000000000000000000 P fbacc683344144319504f35018d9dc1c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fbacc683344144319504f35018d9dc1c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42645 } }
I20260619 10:21:13.136536 22251 leader_election.cc:302] T 00000000000000000000000000000000 P fbacc683344144319504f35018d9dc1c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fbacc683344144319504f35018d9dc1c; no voters:
I20260619 10:21:13.136698 22251 leader_election.cc:288] T 00000000000000000000000000000000 P fbacc683344144319504f35018d9dc1c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 10:21:13.136746 22254 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fbacc683344144319504f35018d9dc1c [term 1 FOLLOWER]: Leader election won for term 1
I20260619 10:21:13.136926 22254 raft_consensus.cc:686] T 00000000000000000000000000000000 P fbacc683344144319504f35018d9dc1c [term 1 LEADER]: Becoming Leader. State: Replica: fbacc683344144319504f35018d9dc1c, State: Running, Role: LEADER
I20260619 10:21:13.136934 22251 sys_catalog.cc:564] T 00000000000000000000000000000000 P fbacc683344144319504f35018d9dc1c [sys.catalog]: configured and running, proceeding with master startup.
I20260619 10:21:13.137051 22254 consensus_queue.cc:237] T 00000000000000000000000000000000 P fbacc683344144319504f35018d9dc1c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fbacc683344144319504f35018d9dc1c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42645 } }
I20260619 10:21:13.137468 22256 sys_catalog.cc:455] T 00000000000000000000000000000000 P fbacc683344144319504f35018d9dc1c [sys.catalog]: SysCatalogTable state changed. Reason: New leader fbacc683344144319504f35018d9dc1c. Latest consensus state: current_term: 1 leader_uuid: "fbacc683344144319504f35018d9dc1c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fbacc683344144319504f35018d9dc1c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42645 } } }
I20260619 10:21:13.137496 22255 sys_catalog.cc:455] T 00000000000000000000000000000000 P fbacc683344144319504f35018d9dc1c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fbacc683344144319504f35018d9dc1c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fbacc683344144319504f35018d9dc1c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42645 } } }
I20260619 10:21:13.137669 22255 sys_catalog.cc:458] T 00000000000000000000000000000000 P fbacc683344144319504f35018d9dc1c [sys.catalog]: This master's current role is: LEADER
I20260619 10:21:13.137629 22256 sys_catalog.cc:458] T 00000000000000000000000000000000 P fbacc683344144319504f35018d9dc1c [sys.catalog]: This master's current role is: LEADER
I20260619 10:21:13.138073 22259 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 10:21:13.138628 22259 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 10:21:13.139076 825 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTU9OP6/build/release/bin/kudu as pid 22183
I20260619 10:21:13.139189 825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781864224577058-825-0/minicluster-data/master-0/wal/instance
I20260619 10:21:13.139408 825 external_mini_cluster.cc:1428] Setting key a85cff0451e6f65efd0ff73c752140a1
I20260619 10:21:13.140287 22259 catalog_manager.cc:1347] Generated new cluster ID: af4f355083314f0bb1384780d3fff47a
I20260619 10:21:13.140338 22259 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 10:21:13.141775 22274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:21:13.139792 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.235:55163 (local address 127.0.0.1:42645)
0619 10:21:13.139996 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0619 10:21:13.140004 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 10:21:13.140044 (+ 40us) server_negotiation.cc:407] Connection header received
0619 10:21:13.140188 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:21:13.140194 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:21:13.140224 (+ 30us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:21:13.140363 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:21:13.141312 (+ 949us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 10:21:13.141315 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 10:21:13.141316 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 10:21:13.141320 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0619 10:21:13.141366 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 10:21:13.141377 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 10:21:13.141444 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0619 10:21:13.141522 (+ 78us) server_negotiation.cc:299] Negotiation successful
0619 10:21:13.141648 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":159,"thread_start_us":121,"threads_started":1}
I20260619 10:21:13.142563 825 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 10:21:13.145049 22274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:21:13.143904 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.235:36939 (local address 127.0.0.1:42645)
0619 10:21:13.143938 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0619 10:21:13.143940 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 10:21:13.144152 (+ 212us) server_negotiation.cc:407] Connection header received
0619 10:21:13.144208 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:21:13.144209 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:21:13.144214 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:21:13.144266 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:21:13.144818 (+ 552us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 10:21:13.144819 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 10:21:13.144820 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 10:21:13.144824 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0619 10:21:13.144838 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 10:21:13.144841 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 10:21:13.144914 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0619 10:21:13.144969 (+ 55us) server_negotiation.cc:299] Negotiation successful
0619 10:21:13.144990 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260619 10:21:13.152503 22259 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 10:21:13.152683 22259 catalog_manager.cc:1504] Loading token signing keys...
I20260619 10:21:13.164122 22259 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fbacc683344144319504f35018d9dc1c: Generated new TSK 0
I20260619 10:21:13.164312 22259 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 10:21:13.168478 825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTU9OP6/build/release/bin/kudu with pid 22183
2026-06-19T10:21:13Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (1271 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781864224577058-825-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 10:21:13 dist-test-slave-6pt2 krb5kdc[22282](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 10:21:13 dist-test-slave-6pt2 krb5kdc[22282](info): set up 2 sockets
Jun 19 10:21:13 dist-test-slave-6pt2 krb5kdc[22282](info): commencing operation
krb5kdc: starting...
W20260619 10:21:14.178792 825 mini_kdc.cc:121] Time spent starting KDC: real 0.993s user 0.002s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781864224577058-825-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781864224577058-825-0/krb5kdc/test-user.keytab.
Jun 19 10:21:14 dist-test-slave-6pt2 krb5kdc[22282](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864474, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T10:21:14Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T10:21:14Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.206.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.206.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.206.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781864224577058-825-0/krb5kdc/kudu_127.0.206.126.keytab.
Entry for principal kudu/127.0.206.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781864224577058-825-0/krb5kdc/kudu_127.0.206.126.keytab.
I20260619 10:21:14.287295 825 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTU9OP6/build/release/bin/kudu
/tmp/dist-test-taskTU9OP6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781864224577058-825-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781864224577058-825-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781864224577058-825-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781864224577058-825-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.206.126:33115
--webserver_interface=127.0.206.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781864224577058-825-0/krb5kdc/kudu_127.0.206.126.keytab
--principal=kudu/127.0.206.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.206.84:37227
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.206.126:33115
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781864224577058-825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781864224577058-825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781864224577058-825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 10:21:14.363386 22295 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 10:21:14.363616 22295 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 10:21:14.363656 22295 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 10:21:14.364784 22295 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 10:21:14.364848 22295 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 10:21:14.364871 22295 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 10:21:14.364890 22295 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 10:21:14.364910 22295 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 10:21:14.366357 22295 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.206.84:37227
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781864224577058-825-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781864224577058-825-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.206.126:33115
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781864224577058-825-0/krb5kdc/kudu_127.0.206.126.keytab
--principal=kudu/127.0.206.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.206.126:33115
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781864224577058-825-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.206.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22295
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781864224577058-825-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 09:32:44 UTC on ccycloud.quasar-gbtldp.root.comops.site
build id 4455440
I20260619 10:21:14.366694 22295 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 10:21:14.366905 22295 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 10:21:14.371423 22300 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 10:21:14.371451 22303 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 10:21:14.371435 22301 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 10:21:14.371732 22295 server_base.cc:1034] running on GCE node
I20260619 10:21:14.371953 22295 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 10:21:14.372166 22295 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 10:21:14.373323 22295 hybrid_clock.cc:648] HybridClock initialized: now 1781864474373305 us; error 46 us; skew 500 ppm
Jun 19 10:21:14 dist-test-slave-6pt2 krb5kdc[22282](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864474, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 10:21:14.378670 22295 init.cc:377] Logged in from keytab as kudu/127.0.206.126@KRBTEST.COM (short username kudu)
I20260619 10:21:14.379053 22295 webserver.cc:460] Webserver started at http://127.0.206.126:42425/ using document root <none> and password file <none>
I20260619 10:21:14.379268 22295 fs_manager.cc:359] Metadata directory not provided
I20260619 10:21:14.379333 22295 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 10:21:14.379478 22295 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 10:21:14.380470 22295 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781864224577058-825-0/minicluster-data/master-0/data/instance:
uuid: "231a8058bc57480997a74988fd020226"
format_stamp: "Formatted at 2026-06-19 10:21:14 on dist-test-slave-6pt2"
server_key: "12db11c739d082dbc767f9b9e71ba47c"
server_key_iv: "4b0dfef653ac36b9df9df9ebab195e5a"
server_key_version: "encryptionkey@0"
I20260619 10:21:14.380859 22295 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781864224577058-825-0/minicluster-data/master-0/wal/instance:
uuid: "231a8058bc57480997a74988fd020226"
format_stamp: "Formatted at 2026-06-19 10:21:14 on dist-test-slave-6pt2"
server_key: "12db11c739d082dbc767f9b9e71ba47c"
server_key_iv: "4b0dfef653ac36b9df9df9ebab195e5a"
server_key_version: "encryptionkey@0"
I20260619 10:21:14.382102 22295 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 10:21:14.382853 22310 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 10:21:14.383011 22295 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 10:21:14.383111 22295 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781864224577058-825-0/minicluster-data/master-0/data,/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781864224577058-825-0/minicluster-data/master-0/wal
uuid: "231a8058bc57480997a74988fd020226"
format_stamp: "Formatted at 2026-06-19 10:21:14 on dist-test-slave-6pt2"
server_key: "12db11c739d082dbc767f9b9e71ba47c"
server_key_iv: "4b0dfef653ac36b9df9df9ebab195e5a"
server_key_version: "encryptionkey@0"
I20260619 10:21:14.383178 22295 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781864224577058-825-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781864224577058-825-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781864224577058-825-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 10:21:14.394394 22295 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 10:21:14.396055 22295 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 10:21:14.396214 22295 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 10:21:14.399817 22295 rpc_server.cc:307] RPC server started. Bound to: 127.0.206.126:33115
I20260619 10:21:14.399875 22362 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.206.126:33115 every 8 connection(s)
I20260619 10:21:14.400163 22295 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781864224577058-825-0/minicluster-data/master-0/data/info.pb
I20260619 10:21:14.401098 22363 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 10:21:14.401202 825 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTU9OP6/build/release/bin/kudu as pid 22295
I20260619 10:21:14.401333 825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781864224577058-825-0/minicluster-data/master-0/wal/instance
I20260619 10:21:14.401551 825 external_mini_cluster.cc:1428] Setting key 38f13bed13faa8f1ed4dd393cd318e56
I20260619 10:21:14.404204 22363 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 231a8058bc57480997a74988fd020226: Bootstrap starting.
I20260619 10:21:14.404841 22363 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 231a8058bc57480997a74988fd020226: Neither blocks nor log segments found. Creating new log.
I20260619 10:21:14.405153 22363 log.cc:824] T 00000000000000000000000000000000 P 231a8058bc57480997a74988fd020226: Log is configured to *not* fsync() on all Append() calls
I20260619 10:21:14.405766 22363 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 231a8058bc57480997a74988fd020226: No bootstrap required, opened a new log
I20260619 10:21:14.407534 22363 raft_consensus.cc:348] T 00000000000000000000000000000000 P 231a8058bc57480997a74988fd020226 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "231a8058bc57480997a74988fd020226" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 33115 } }
I20260619 10:21:14.407648 22363 raft_consensus.cc:374] T 00000000000000000000000000000000 P 231a8058bc57480997a74988fd020226 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 10:21:14.407670 22363 raft_consensus.cc:729] T 00000000000000000000000000000000 P 231a8058bc57480997a74988fd020226 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 231a8058bc57480997a74988fd020226, State: Initialized, Role: FOLLOWER
I20260619 10:21:14.407797 22363 consensus_queue.cc:260] T 00000000000000000000000000000000 P 231a8058bc57480997a74988fd020226 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "231a8058bc57480997a74988fd020226" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 33115 } }
I20260619 10:21:14.407852 22363 raft_consensus.cc:388] T 00000000000000000000000000000000 P 231a8058bc57480997a74988fd020226 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 10:21:14.407900 22363 raft_consensus.cc:482] T 00000000000000000000000000000000 P 231a8058bc57480997a74988fd020226 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
Jun 19 10:21:14 dist-test-slave-6pt2 krb5kdc[22282](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864474, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.206.126@KRBTEST.COM
I20260619 10:21:14.407964 22363 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 231a8058bc57480997a74988fd020226 [term 0 FOLLOWER]: Advancing to term 1
I20260619 10:21:14.408646 22363 raft_consensus.cc:504] T 00000000000000000000000000000000 P 231a8058bc57480997a74988fd020226 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "231a8058bc57480997a74988fd020226" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 33115 } }
I20260619 10:21:14.408728 22363 leader_election.cc:302] T 00000000000000000000000000000000 P 231a8058bc57480997a74988fd020226 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 231a8058bc57480997a74988fd020226; no voters:
I20260619 10:21:14.408905 22363 leader_election.cc:288] T 00000000000000000000000000000000 P 231a8058bc57480997a74988fd020226 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 10:21:14.408973 22368 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 231a8058bc57480997a74988fd020226 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 10:21:14.409135 22363 sys_catalog.cc:564] T 00000000000000000000000000000000 P 231a8058bc57480997a74988fd020226 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 10:21:14.409135 22368 raft_consensus.cc:686] T 00000000000000000000000000000000 P 231a8058bc57480997a74988fd020226 [term 1 LEADER]: Becoming Leader. State: Replica: 231a8058bc57480997a74988fd020226, State: Running, Role: LEADER
I20260619 10:21:14.409271 22368 consensus_queue.cc:237] T 00000000000000000000000000000000 P 231a8058bc57480997a74988fd020226 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "231a8058bc57480997a74988fd020226" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 33115 } }
I20260619 10:21:14.409931 22369 sys_catalog.cc:455] T 00000000000000000000000000000000 P 231a8058bc57480997a74988fd020226 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 231a8058bc57480997a74988fd020226. Latest consensus state: current_term: 1 leader_uuid: "231a8058bc57480997a74988fd020226" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "231a8058bc57480997a74988fd020226" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 33115 } } }
I20260619 10:21:14.410007 22369 sys_catalog.cc:458] T 00000000000000000000000000000000 P 231a8058bc57480997a74988fd020226 [sys.catalog]: This master's current role is: LEADER
I20260619 10:21:14.410501 22377 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 10:21:14.410135 22370 sys_catalog.cc:455] T 00000000000000000000000000000000 P 231a8058bc57480997a74988fd020226 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "231a8058bc57480997a74988fd020226" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "231a8058bc57480997a74988fd020226" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 33115 } } }
I20260619 10:21:14.410928 22370 sys_catalog.cc:458] T 00000000000000000000000000000000 P 231a8058bc57480997a74988fd020226 [sys.catalog]: This master's current role is: LEADER
I20260619 10:21:14.411358 22377 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 10:21:14.412428 22366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:21:14.402017 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36115 (local address 127.0.206.126:33115)
0619 10:21:14.402185 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0619 10:21:14.402193 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 10:21:14.402223 (+ 30us) server_negotiation.cc:407] Connection header received
0619 10:21:14.402689 (+ 466us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:21:14.402696 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:21:14.402749 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:21:14.402851 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:21:14.403580 (+ 729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:21:14.404441 (+ 861us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:21:14.405536 (+ 1095us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:21:14.405991 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:21:14.406100 (+ 109us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:21:14.408516 (+ 2416us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 10:21:14.408520 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 10:21:14.408520 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 10:21:14.408529 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0619 10:21:14.410638 (+ 2109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:21:14.411092 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:21:14.411095 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:21:14.411096 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:21:14.411210 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:21:14.411469 (+ 259us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:21:14.411470 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:21:14.411470 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:21:14.411750 (+ 280us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 10:21:14.411884 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0619 10:21:14.412139 (+ 255us) server_negotiation.cc:299] Negotiation successful
0619 10:21:14.412236 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":129,"thread_start_us":88,"threads_started":1}
I20260619 10:21:14.412811 22377 catalog_manager.cc:1347] Generated new cluster ID: 1a9ed84a84db444f8af77bca60e34352
I20260619 10:21:14.412866 22377 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 10:21:14.413424 825 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 10:21:14.421854 22366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:21:14.415027 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54529 (local address 127.0.206.126:33115)
0619 10:21:14.415069 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0619 10:21:14.415071 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 10:21:14.415162 (+ 91us) server_negotiation.cc:407] Connection header received
0619 10:21:14.415187 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:21:14.415188 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:21:14.415199 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:21:14.415265 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:21:14.415818 (+ 553us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:21:14.416604 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:21:14.417540 (+ 936us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:21:14.417876 (+ 336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:21:14.417950 (+ 74us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:21:14.419213 (+ 1263us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 10:21:14.419214 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 10:21:14.419215 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 10:21:14.419221 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 10:21:14.420701 (+ 1480us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:21:14.421070 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:21:14.421072 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:21:14.421072 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:21:14.421129 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:21:14.421341 (+ 212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:21:14.421342 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:21:14.421342 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:21:14.421492 (+ 150us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 10:21:14.421597 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0619 10:21:14.421714 (+ 117us) server_negotiation.cc:299] Negotiation successful
0619 10:21:14.421772 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260619 10:21:14.440375 22377 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 10:21:14.441037 22377 catalog_manager.cc:1504] Loading token signing keys...
I20260619 10:21:14.449556 22377 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 231a8058bc57480997a74988fd020226: Generated new TSK 0
I20260619 10:21:14.449710 22377 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 10:21:14.452389 825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTU9OP6/build/release/bin/kudu with pid 22295
W20260619 10:21:14.456753 22280 connection.cc:535] client connection to 127.0.206.126:33115 recv error: Network error: failed to read from TLS socket (remote: 127.0.206.126:33115): Connection reset by peer (error 104)
2026-06-19T10:21:14Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (1284 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781864224577058-825-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 10:21:14 dist-test-slave-6pt2 krb5kdc[22394](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 10:21:14 dist-test-slave-6pt2 krb5kdc[22394](info): set up 2 sockets
Jun 19 10:21:14 dist-test-slave-6pt2 krb5kdc[22394](info): commencing operation
krb5kdc: starting...
W20260619 10:21:15.474644 825 mini_kdc.cc:121] Time spent starting KDC: real 1.004s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781864224577058-825-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781864224577058-825-0/krb5kdc/test-user.keytab.
Jun 19 10:21:15 dist-test-slave-6pt2 krb5kdc[22394](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864475, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T10:21:15Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T10:21:15Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.206.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.206.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.206.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781864224577058-825-0/krb5kdc/kudu_127.0.206.126.keytab.
Entry for principal kudu/127.0.206.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781864224577058-825-0/krb5kdc/kudu_127.0.206.126.keytab.
I20260619 10:21:15.584821 825 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTU9OP6/build/release/bin/kudu
/tmp/dist-test-taskTU9OP6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781864224577058-825-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781864224577058-825-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781864224577058-825-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781864224577058-825-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.206.126:38487
--webserver_interface=127.0.206.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781864224577058-825-0/krb5kdc/kudu_127.0.206.126.keytab
--principal=kudu/127.0.206.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.206.84:42737
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.206.126:38487
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781864224577058-825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781864224577058-825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781864224577058-825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 10:21:15.660921 22407 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 10:21:15.661127 22407 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 10:21:15.661149 22407 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 10:21:15.662158 22407 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 10:21:15.662189 22407 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 10:21:15.662202 22407 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 10:21:15.662214 22407 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 10:21:15.662225 22407 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 10:21:15.663538 22407 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.206.84:42737
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781864224577058-825-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781864224577058-825-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.206.126:38487
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781864224577058-825-0/krb5kdc/kudu_127.0.206.126.keytab
--principal=kudu/127.0.206.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.206.126:38487
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781864224577058-825-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.206.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22407
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781864224577058-825-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 09:32:44 UTC on ccycloud.quasar-gbtldp.root.comops.site
build id 4455440
I20260619 10:21:15.663704 22407 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 10:21:15.663854 22407 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 10:21:15.668349 22412 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 10:21:15.668349 22415 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 10:21:15.668340 22413 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 10:21:15.668574 22407 server_base.cc:1034] running on GCE node
I20260619 10:21:15.668792 22407 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 10:21:15.669062 22407 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 10:21:15.670220 22407 hybrid_clock.cc:648] HybridClock initialized: now 1781864475670214 us; error 33 us; skew 500 ppm
Jun 19 10:21:15 dist-test-slave-6pt2 krb5kdc[22394](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864475, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 10:21:15.672540 22407 init.cc:377] Logged in from keytab as kudu/127.0.206.126@KRBTEST.COM (short username kudu)
I20260619 10:21:15.672969 22407 webserver.cc:460] Webserver started at http://127.0.206.126:34743/ using document root <none> and password file <none>
I20260619 10:21:15.673193 22407 fs_manager.cc:359] Metadata directory not provided
I20260619 10:21:15.673257 22407 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 10:21:15.673408 22407 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 10:21:15.674363 22407 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781864224577058-825-0/minicluster-data/master-0/data/instance:
uuid: "59fb72c199bc42dc9ebfa2e3c3d29365"
format_stamp: "Formatted at 2026-06-19 10:21:15 on dist-test-slave-6pt2"
server_key: "30b4463b8d04df9400309f9a9a6fa185"
server_key_iv: "11fd63929c408df08d3bf7041971a325"
server_key_version: "encryptionkey@0"
I20260619 10:21:15.674733 22407 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781864224577058-825-0/minicluster-data/master-0/wal/instance:
uuid: "59fb72c199bc42dc9ebfa2e3c3d29365"
format_stamp: "Formatted at 2026-06-19 10:21:15 on dist-test-slave-6pt2"
server_key: "30b4463b8d04df9400309f9a9a6fa185"
server_key_iv: "11fd63929c408df08d3bf7041971a325"
server_key_version: "encryptionkey@0"
I20260619 10:21:15.676028 22407 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260619 10:21:15.676765 22422 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 10:21:15.676931 22407 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260619 10:21:15.677026 22407 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781864224577058-825-0/minicluster-data/master-0/data,/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781864224577058-825-0/minicluster-data/master-0/wal
uuid: "59fb72c199bc42dc9ebfa2e3c3d29365"
format_stamp: "Formatted at 2026-06-19 10:21:15 on dist-test-slave-6pt2"
server_key: "30b4463b8d04df9400309f9a9a6fa185"
server_key_iv: "11fd63929c408df08d3bf7041971a325"
server_key_version: "encryptionkey@0"
I20260619 10:21:15.677110 22407 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781864224577058-825-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781864224577058-825-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781864224577058-825-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 10:21:15.688611 22407 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 10:21:15.690224 22407 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 10:21:15.690383 22407 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 10:21:15.694144 22407 rpc_server.cc:307] RPC server started. Bound to: 127.0.206.126:38487
I20260619 10:21:15.694177 22474 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.206.126:38487 every 8 connection(s)
I20260619 10:21:15.694511 22407 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781864224577058-825-0/minicluster-data/master-0/data/info.pb
I20260619 10:21:15.695343 22475 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 10:21:15.697458 22475 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 59fb72c199bc42dc9ebfa2e3c3d29365: Bootstrap starting.
I20260619 10:21:15.697997 22475 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 59fb72c199bc42dc9ebfa2e3c3d29365: Neither blocks nor log segments found. Creating new log.
I20260619 10:21:15.698268 22475 log.cc:824] T 00000000000000000000000000000000 P 59fb72c199bc42dc9ebfa2e3c3d29365: Log is configured to *not* fsync() on all Append() calls
I20260619 10:21:15.698812 825 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTU9OP6/build/release/bin/kudu as pid 22407
I20260619 10:21:15.698922 825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781864224577058-825-0/minicluster-data/master-0/wal/instance
I20260619 10:21:15.699095 22475 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 59fb72c199bc42dc9ebfa2e3c3d29365: No bootstrap required, opened a new log
I20260619 10:21:15.699115 825 external_mini_cluster.cc:1428] Setting key 1a9e6c11a72ef5be2a1ab5b0b0458baf
I20260619 10:21:15.701614 22475 raft_consensus.cc:348] T 00000000000000000000000000000000 P 59fb72c199bc42dc9ebfa2e3c3d29365 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "59fb72c199bc42dc9ebfa2e3c3d29365" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 38487 } }
I20260619 10:21:15.701792 22475 raft_consensus.cc:374] T 00000000000000000000000000000000 P 59fb72c199bc42dc9ebfa2e3c3d29365 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 10:21:15.701830 22475 raft_consensus.cc:729] T 00000000000000000000000000000000 P 59fb72c199bc42dc9ebfa2e3c3d29365 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 59fb72c199bc42dc9ebfa2e3c3d29365, State: Initialized, Role: FOLLOWER
I20260619 10:21:15.701949 22475 consensus_queue.cc:260] T 00000000000000000000000000000000 P 59fb72c199bc42dc9ebfa2e3c3d29365 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "59fb72c199bc42dc9ebfa2e3c3d29365" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 38487 } }
I20260619 10:21:15.702026 22475 raft_consensus.cc:388] T 00000000000000000000000000000000 P 59fb72c199bc42dc9ebfa2e3c3d29365 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 10:21:15.702056 22475 raft_consensus.cc:482] T 00000000000000000000000000000000 P 59fb72c199bc42dc9ebfa2e3c3d29365 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 10:21:15.702101 22475 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 59fb72c199bc42dc9ebfa2e3c3d29365 [term 0 FOLLOWER]: Advancing to term 1
I20260619 10:21:15.702761 22475 raft_consensus.cc:504] T 00000000000000000000000000000000 P 59fb72c199bc42dc9ebfa2e3c3d29365 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "59fb72c199bc42dc9ebfa2e3c3d29365" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 38487 } }
I20260619 10:21:15.702857 22475 leader_election.cc:302] T 00000000000000000000000000000000 P 59fb72c199bc42dc9ebfa2e3c3d29365 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 59fb72c199bc42dc9ebfa2e3c3d29365; no voters:
I20260619 10:21:15.703073 22475 leader_election.cc:288] T 00000000000000000000000000000000 P 59fb72c199bc42dc9ebfa2e3c3d29365 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 10:21:15.703094 22480 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 59fb72c199bc42dc9ebfa2e3c3d29365 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 10:21:15.703202 22480 raft_consensus.cc:686] T 00000000000000000000000000000000 P 59fb72c199bc42dc9ebfa2e3c3d29365 [term 1 LEADER]: Becoming Leader. State: Replica: 59fb72c199bc42dc9ebfa2e3c3d29365, State: Running, Role: LEADER
I20260619 10:21:15.703353 22475 sys_catalog.cc:564] T 00000000000000000000000000000000 P 59fb72c199bc42dc9ebfa2e3c3d29365 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 10:21:15.703334 22480 consensus_queue.cc:237] T 00000000000000000000000000000000 P 59fb72c199bc42dc9ebfa2e3c3d29365 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "59fb72c199bc42dc9ebfa2e3c3d29365" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 38487 } }
I20260619 10:21:15.703744 22481 sys_catalog.cc:455] T 00000000000000000000000000000000 P 59fb72c199bc42dc9ebfa2e3c3d29365 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "59fb72c199bc42dc9ebfa2e3c3d29365" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "59fb72c199bc42dc9ebfa2e3c3d29365" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 38487 } } }
I20260619 10:21:15.703969 22481 sys_catalog.cc:458] T 00000000000000000000000000000000 P 59fb72c199bc42dc9ebfa2e3c3d29365 [sys.catalog]: This master's current role is: LEADER
I20260619 10:21:15.703902 22482 sys_catalog.cc:455] T 00000000000000000000000000000000 P 59fb72c199bc42dc9ebfa2e3c3d29365 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 59fb72c199bc42dc9ebfa2e3c3d29365. Latest consensus state: current_term: 1 leader_uuid: "59fb72c199bc42dc9ebfa2e3c3d29365" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "59fb72c199bc42dc9ebfa2e3c3d29365" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 38487 } } }
I20260619 10:21:15.704244 22482 sys_catalog.cc:458] T 00000000000000000000000000000000 P 59fb72c199bc42dc9ebfa2e3c3d29365 [sys.catalog]: This master's current role is: LEADER
I20260619 10:21:15.704442 22485 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 10:21:15.704893 22485 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 10:21:15.704898 22479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:21:15.699587 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50193 (local address 127.0.206.126:38487)
0619 10:21:15.699772 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0619 10:21:15.699781 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 10:21:15.699813 (+ 32us) server_negotiation.cc:407] Connection header received
0619 10:21:15.700353 (+ 540us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:21:15.700359 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:21:15.700412 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:21:15.700525 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:21:15.700736 (+ 211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:21:15.701607 (+ 871us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:21:15.702744 (+ 1137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:21:15.703334 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:21:15.703725 (+ 391us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:21:15.704118 (+ 393us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 10:21:15.704123 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 10:21:15.704123 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 10:21:15.704321 (+ 198us) server_negotiation.cc:922] calling sasl_server_start()
0619 10:21:15.704360 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 10:21:15.704374 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 10:21:15.704454 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0619 10:21:15.704636 (+ 182us) server_negotiation.cc:299] Negotiation successful
0619 10:21:15.704663 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":138,"thread_start_us":100,"threads_started":1}
I20260619 10:21:15.706326 825 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 10:21:15.706683 22485 catalog_manager.cc:1347] Generated new cluster ID: 7140dbb10d4043cfa959bbde690d8d6e
I20260619 10:21:15.706732 22485 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 10:21:15.710421 22485 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 10:21:15.711081 22485 catalog_manager.cc:1504] Loading token signing keys...
I20260619 10:21:15.714309 22479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:21:15.710886 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36903 (local address 127.0.206.126:38487)
0619 10:21:15.710920 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0619 10:21:15.710922 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 10:21:15.711076 (+ 154us) server_negotiation.cc:407] Connection header received
0619 10:21:15.711132 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:21:15.711133 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:21:15.711145 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:21:15.711200 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:21:15.711347 (+ 147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:21:15.712096 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:21:15.713056 (+ 960us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:21:15.713354 (+ 298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:21:15.713422 (+ 68us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:21:15.714080 (+ 658us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 10:21:15.714081 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 10:21:15.714082 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 10:21:15.714086 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0619 10:21:15.714099 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 10:21:15.714103 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 10:21:15.714164 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0619 10:21:15.714230 (+ 66us) server_negotiation.cc:299] Negotiation successful
0619 10:21:15.714239 (+ 9us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260619 10:21:15.715427 22485 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 59fb72c199bc42dc9ebfa2e3c3d29365: Generated new TSK 0
I20260619 10:21:15.715577 22485 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 10:21:15.719762 825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTU9OP6/build/release/bin/kudu with pid 22407
W20260619 10:21:15.723714 22392 connection.cc:535] client connection to 127.0.206.126:38487 recv error: Network error: failed to read from TLS socket (remote: 127.0.206.126:38487): Connection reset by peer (error 104)
2026-06-19T10:21:15Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (1267 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781864224577058-825-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 10:21:15 dist-test-slave-6pt2 krb5kdc[22506](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 10:21:15 dist-test-slave-6pt2 krb5kdc[22506](info): set up 2 sockets
Jun 19 10:21:15 dist-test-slave-6pt2 krb5kdc[22506](info): commencing operation
krb5kdc: starting...
W20260619 10:21:16.726670 825 mini_kdc.cc:121] Time spent starting KDC: real 0.989s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781864224577058-825-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781864224577058-825-0/krb5kdc/test-user.keytab.
Jun 19 10:21:16 dist-test-slave-6pt2 krb5kdc[22506](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864476, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T10:21:16Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T10:21:16Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.206.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.206.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.206.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781864224577058-825-0/krb5kdc/kudu_127.0.206.126.keytab.
Entry for principal kudu/127.0.206.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781864224577058-825-0/krb5kdc/kudu_127.0.206.126.keytab.
I20260619 10:21:16.833338 825 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTU9OP6/build/release/bin/kudu
/tmp/dist-test-taskTU9OP6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781864224577058-825-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781864224577058-825-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781864224577058-825-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781864224577058-825-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.206.126:42595
--webserver_interface=127.0.206.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781864224577058-825-0/krb5kdc/kudu_127.0.206.126.keytab
--principal=kudu/127.0.206.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.206.84:37001
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.206.126:42595
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781864224577058-825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781864224577058-825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781864224577058-825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 10:21:16.910753 22519 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 10:21:16.910956 22519 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 10:21:16.910979 22519 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 10:21:16.912096 22519 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 10:21:16.912137 22519 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 10:21:16.912150 22519 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 10:21:16.912206 22519 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 10:21:16.912235 22519 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 10:21:16.913621 22519 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.206.84:37001
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781864224577058-825-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781864224577058-825-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.206.126:42595
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781864224577058-825-0/krb5kdc/kudu_127.0.206.126.keytab
--principal=kudu/127.0.206.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.206.126:42595
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781864224577058-825-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.206.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22519
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781864224577058-825-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 09:32:44 UTC on ccycloud.quasar-gbtldp.root.comops.site
build id 4455440
I20260619 10:21:16.913808 22519 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 10:21:16.914003 22519 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 10:21:16.918550 22525 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 10:21:16.918629 22519 server_base.cc:1034] running on GCE node
W20260619 10:21:16.918547 22524 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 10:21:16.918550 22527 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 10:21:16.918975 22519 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 10:21:16.919258 22519 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 10:21:16.920423 22519 hybrid_clock.cc:648] HybridClock initialized: now 1781864476920416 us; error 34 us; skew 500 ppm
Jun 19 10:21:16 dist-test-slave-6pt2 krb5kdc[22506](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864476, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 10:21:16.922796 22519 init.cc:377] Logged in from keytab as kudu/127.0.206.126@KRBTEST.COM (short username kudu)
I20260619 10:21:16.923233 22519 webserver.cc:460] Webserver started at http://127.0.206.126:35719/ using document root <none> and password file <none>
I20260619 10:21:16.923435 22519 fs_manager.cc:359] Metadata directory not provided
I20260619 10:21:16.923499 22519 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 10:21:16.923646 22519 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 10:21:16.924634 22519 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781864224577058-825-0/minicluster-data/master-0/data/instance:
uuid: "e2e032ed4f0c4151af28d214f6ac9bfc"
format_stamp: "Formatted at 2026-06-19 10:21:16 on dist-test-slave-6pt2"
server_key: "ebf8c0e6d66f3a68a229837e3f7574a5"
server_key_iv: "70579f8b9ad3c16600de46b20120d8f6"
server_key_version: "encryptionkey@0"
I20260619 10:21:16.924989 22519 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781864224577058-825-0/minicluster-data/master-0/wal/instance:
uuid: "e2e032ed4f0c4151af28d214f6ac9bfc"
format_stamp: "Formatted at 2026-06-19 10:21:16 on dist-test-slave-6pt2"
server_key: "ebf8c0e6d66f3a68a229837e3f7574a5"
server_key_iv: "70579f8b9ad3c16600de46b20120d8f6"
server_key_version: "encryptionkey@0"
I20260619 10:21:16.926224 22519 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 10:21:16.927021 22534 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 10:21:16.927186 22519 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260619 10:21:16.927280 22519 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781864224577058-825-0/minicluster-data/master-0/data,/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781864224577058-825-0/minicluster-data/master-0/wal
uuid: "e2e032ed4f0c4151af28d214f6ac9bfc"
format_stamp: "Formatted at 2026-06-19 10:21:16 on dist-test-slave-6pt2"
server_key: "ebf8c0e6d66f3a68a229837e3f7574a5"
server_key_iv: "70579f8b9ad3c16600de46b20120d8f6"
server_key_version: "encryptionkey@0"
I20260619 10:21:16.927358 22519 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781864224577058-825-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781864224577058-825-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781864224577058-825-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 10:21:16.930333 22519 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 10:21:16.932513 22519 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 10:21:16.932657 22519 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 10:21:16.936396 22519 rpc_server.cc:307] RPC server started. Bound to: 127.0.206.126:42595
I20260619 10:21:16.936425 22586 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.206.126:42595 every 8 connection(s)
I20260619 10:21:16.936746 22519 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781864224577058-825-0/minicluster-data/master-0/data/info.pb
I20260619 10:21:16.937248 825 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTU9OP6/build/release/bin/kudu as pid 22519
I20260619 10:21:16.937382 825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781864224577058-825-0/minicluster-data/master-0/wal/instance
I20260619 10:21:16.937572 825 external_mini_cluster.cc:1428] Setting key c1d2eaccfc4510428803a954155f5e8f
I20260619 10:21:16.937841 22587 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 10:21:16.940116 22590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:21:16.938019 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59229 (local address 127.0.206.126:42595)
0619 10:21:16.938209 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0619 10:21:16.938217 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 10:21:16.938248 (+ 31us) server_negotiation.cc:407] Connection header received
0619 10:21:16.938941 (+ 693us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:21:16.938950 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:21:16.938983 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:21:16.939132 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:21:16.939728 (+ 596us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 10:21:16.939732 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 10:21:16.939732 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 10:21:16.939736 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0619 10:21:16.939777 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 10:21:16.939789 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 10:21:16.939865 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0619 10:21:16.939900 (+ 35us) server_negotiation.cc:299] Negotiation successful
0619 10:21:16.939937 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":136,"thread_start_us":86,"threads_started":1}
I20260619 10:21:16.941068 825 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 10:21:16.941184 22587 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e2e032ed4f0c4151af28d214f6ac9bfc: Bootstrap starting.
I20260619 10:21:16.941902 22587 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e2e032ed4f0c4151af28d214f6ac9bfc: Neither blocks nor log segments found. Creating new log.
I20260619 10:21:16.942198 22587 log.cc:824] T 00000000000000000000000000000000 P e2e032ed4f0c4151af28d214f6ac9bfc: Log is configured to *not* fsync() on all Append() calls
I20260619 10:21:16.942883 22587 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e2e032ed4f0c4151af28d214f6ac9bfc: No bootstrap required, opened a new log
I20260619 10:21:16.943691 22590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:21:16.942659 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39769 (local address 127.0.206.126:42595)
0619 10:21:16.942699 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0619 10:21:16.942701 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 10:21:16.942810 (+ 109us) server_negotiation.cc:407] Connection header received
0619 10:21:16.942876 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:21:16.942877 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:21:16.942882 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:21:16.942937 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:21:16.943469 (+ 532us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 10:21:16.943470 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 10:21:16.943471 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 10:21:16.943475 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0619 10:21:16.943485 (+ 10us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 10:21:16.943489 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 10:21:16.943542 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0619 10:21:16.943616 (+ 74us) server_negotiation.cc:299] Negotiation successful
0619 10:21:16.943629 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260619 10:21:16.945025 22587 raft_consensus.cc:348] T 00000000000000000000000000000000 P e2e032ed4f0c4151af28d214f6ac9bfc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e2e032ed4f0c4151af28d214f6ac9bfc" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 42595 } }
I20260619 10:21:16.945148 22587 raft_consensus.cc:374] T 00000000000000000000000000000000 P e2e032ed4f0c4151af28d214f6ac9bfc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 10:21:16.945170 22587 raft_consensus.cc:729] T 00000000000000000000000000000000 P e2e032ed4f0c4151af28d214f6ac9bfc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e2e032ed4f0c4151af28d214f6ac9bfc, State: Initialized, Role: FOLLOWER
I20260619 10:21:16.945308 22587 consensus_queue.cc:260] T 00000000000000000000000000000000 P e2e032ed4f0c4151af28d214f6ac9bfc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e2e032ed4f0c4151af28d214f6ac9bfc" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 42595 } }
I20260619 10:21:16.945375 22587 raft_consensus.cc:388] T 00000000000000000000000000000000 P e2e032ed4f0c4151af28d214f6ac9bfc [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 10:21:16.945397 22587 raft_consensus.cc:482] T 00000000000000000000000000000000 P e2e032ed4f0c4151af28d214f6ac9bfc [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 10:21:16.945464 22587 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e2e032ed4f0c4151af28d214f6ac9bfc [term 0 FOLLOWER]: Advancing to term 1
I20260619 10:21:16.945981 22587 raft_consensus.cc:504] T 00000000000000000000000000000000 P e2e032ed4f0c4151af28d214f6ac9bfc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e2e032ed4f0c4151af28d214f6ac9bfc" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 42595 } }
I20260619 10:21:16.946082 22587 leader_election.cc:302] T 00000000000000000000000000000000 P e2e032ed4f0c4151af28d214f6ac9bfc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e2e032ed4f0c4151af28d214f6ac9bfc; no voters:
I20260619 10:21:16.946274 22587 leader_election.cc:288] T 00000000000000000000000000000000 P e2e032ed4f0c4151af28d214f6ac9bfc [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 10:21:16.946345 22597 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e2e032ed4f0c4151af28d214f6ac9bfc [term 1 FOLLOWER]: Leader election won for term 1
I20260619 10:21:16.946527 22587 sys_catalog.cc:564] T 00000000000000000000000000000000 P e2e032ed4f0c4151af28d214f6ac9bfc [sys.catalog]: configured and running, proceeding with master startup.
I20260619 10:21:16.946537 22597 raft_consensus.cc:686] T 00000000000000000000000000000000 P e2e032ed4f0c4151af28d214f6ac9bfc [term 1 LEADER]: Becoming Leader. State: Replica: e2e032ed4f0c4151af28d214f6ac9bfc, State: Running, Role: LEADER
I20260619 10:21:16.946666 22597 consensus_queue.cc:237] T 00000000000000000000000000000000 P e2e032ed4f0c4151af28d214f6ac9bfc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e2e032ed4f0c4151af28d214f6ac9bfc" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 42595 } }
I20260619 10:21:16.947037 22599 sys_catalog.cc:455] T 00000000000000000000000000000000 P e2e032ed4f0c4151af28d214f6ac9bfc [sys.catalog]: SysCatalogTable state changed. Reason: New leader e2e032ed4f0c4151af28d214f6ac9bfc. Latest consensus state: current_term: 1 leader_uuid: "e2e032ed4f0c4151af28d214f6ac9bfc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e2e032ed4f0c4151af28d214f6ac9bfc" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 42595 } } }
I20260619 10:21:16.947139 22599 sys_catalog.cc:458] T 00000000000000000000000000000000 P e2e032ed4f0c4151af28d214f6ac9bfc [sys.catalog]: This master's current role is: LEADER
I20260619 10:21:16.947338 22606 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 10:21:16.947897 22606 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 10:21:16.947356 22598 sys_catalog.cc:455] T 00000000000000000000000000000000 P e2e032ed4f0c4151af28d214f6ac9bfc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e2e032ed4f0c4151af28d214f6ac9bfc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e2e032ed4f0c4151af28d214f6ac9bfc" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 42595 } } }
I20260619 10:21:16.948467 22598 sys_catalog.cc:458] T 00000000000000000000000000000000 P e2e032ed4f0c4151af28d214f6ac9bfc [sys.catalog]: This master's current role is: LEADER
I20260619 10:21:16.949261 22606 catalog_manager.cc:1347] Generated new cluster ID: 838bda09060a426d88d2f974781d79f9
I20260619 10:21:16.949309 22606 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 10:21:16.964939 22606 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 10:21:16.965112 22606 catalog_manager.cc:1504] Loading token signing keys...
I20260619 10:21:16.975641 22606 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e2e032ed4f0c4151af28d214f6ac9bfc: Generated new TSK 0
I20260619 10:21:16.975824 22606 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 10:21:16.979192 825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTU9OP6/build/release/bin/kudu with pid 22519
2026-06-19T10:21:16Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (1259 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781864224577058-825-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 10:21:17 dist-test-slave-6pt2 krb5kdc[22618](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 10:21:17 dist-test-slave-6pt2 krb5kdc[22618](info): set up 2 sockets
Jun 19 10:21:17 dist-test-slave-6pt2 krb5kdc[22618](info): commencing operation
krb5kdc: starting...
W20260619 10:21:17.990803 825 mini_kdc.cc:121] Time spent starting KDC: real 0.994s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781864224577058-825-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781864224577058-825-0/krb5kdc/test-user.keytab.
Jun 19 10:21:18 dist-test-slave-6pt2 krb5kdc[22618](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864478, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T10:21:18Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T10:21:18Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.206.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.206.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.206.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781864224577058-825-0/krb5kdc/kudu_127.0.206.126.keytab.
Entry for principal kudu/127.0.206.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781864224577058-825-0/krb5kdc/kudu_127.0.206.126.keytab.
I20260619 10:21:18.099251 825 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTU9OP6/build/release/bin/kudu
/tmp/dist-test-taskTU9OP6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781864224577058-825-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781864224577058-825-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781864224577058-825-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781864224577058-825-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.206.126:40651
--webserver_interface=127.0.206.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781864224577058-825-0/krb5kdc/kudu_127.0.206.126.keytab
--principal=kudu/127.0.206.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.206.84:46035
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.206.126:40651
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781864224577058-825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781864224577058-825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781864224577058-825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 10:21:18.176218 22631 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 10:21:18.176412 22631 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 10:21:18.176435 22631 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 10:21:18.177563 22631 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 10:21:18.177603 22631 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 10:21:18.177618 22631 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 10:21:18.177630 22631 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 10:21:18.177642 22631 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 10:21:18.179011 22631 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.206.84:46035
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781864224577058-825-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781864224577058-825-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.206.126:40651
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781864224577058-825-0/krb5kdc/kudu_127.0.206.126.keytab
--principal=kudu/127.0.206.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.206.126:40651
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781864224577058-825-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.206.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22631
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781864224577058-825-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 09:32:44 UTC on ccycloud.quasar-gbtldp.root.comops.site
build id 4455440
I20260619 10:21:18.179181 22631 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 10:21:18.179337 22631 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 10:21:18.183781 22639 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 10:21:18.183825 22631 server_base.cc:1034] running on GCE node
W20260619 10:21:18.183775 22636 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 10:21:18.183779 22637 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 10:21:18.184291 22631 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 10:21:18.184583 22631 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 10:21:18.185724 22631 hybrid_clock.cc:648] HybridClock initialized: now 1781864478185710 us; error 30 us; skew 500 ppm
Jun 19 10:21:18 dist-test-slave-6pt2 krb5kdc[22618](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864478, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 10:21:18.188064 22631 init.cc:377] Logged in from keytab as kudu/127.0.206.126@KRBTEST.COM (short username kudu)
I20260619 10:21:18.188510 22631 webserver.cc:460] Webserver started at http://127.0.206.126:38951/ using document root <none> and password file <none>
I20260619 10:21:18.188735 22631 fs_manager.cc:359] Metadata directory not provided
I20260619 10:21:18.188778 22631 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 10:21:18.188935 22631 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 10:21:18.189930 22631 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781864224577058-825-0/minicluster-data/master-0/data/instance:
uuid: "b4023f792f484136a49b2b8fd6accc05"
format_stamp: "Formatted at 2026-06-19 10:21:18 on dist-test-slave-6pt2"
server_key: "732b589ec28c3378a6b72bd1aa42b83f"
server_key_iv: "1e7f9a25c19a2da7ae2651893bcef901"
server_key_version: "encryptionkey@0"
I20260619 10:21:18.190268 22631 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781864224577058-825-0/minicluster-data/master-0/wal/instance:
uuid: "b4023f792f484136a49b2b8fd6accc05"
format_stamp: "Formatted at 2026-06-19 10:21:18 on dist-test-slave-6pt2"
server_key: "732b589ec28c3378a6b72bd1aa42b83f"
server_key_iv: "1e7f9a25c19a2da7ae2651893bcef901"
server_key_version: "encryptionkey@0"
I20260619 10:21:18.191569 22631 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260619 10:21:18.192296 22646 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 10:21:18.192480 22631 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260619 10:21:18.192551 22631 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781864224577058-825-0/minicluster-data/master-0/data,/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781864224577058-825-0/minicluster-data/master-0/wal
uuid: "b4023f792f484136a49b2b8fd6accc05"
format_stamp: "Formatted at 2026-06-19 10:21:18 on dist-test-slave-6pt2"
server_key: "732b589ec28c3378a6b72bd1aa42b83f"
server_key_iv: "1e7f9a25c19a2da7ae2651893bcef901"
server_key_version: "encryptionkey@0"
I20260619 10:21:18.192639 22631 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781864224577058-825-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781864224577058-825-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781864224577058-825-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 10:21:18.211714 22631 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 10:21:18.213369 22631 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 10:21:18.213552 22631 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 10:21:18.217597 22631 rpc_server.cc:307] RPC server started. Bound to: 127.0.206.126:40651
I20260619 10:21:18.217670 22698 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.206.126:40651 every 8 connection(s)
I20260619 10:21:18.217983 22631 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781864224577058-825-0/minicluster-data/master-0/data/info.pb
I20260619 10:21:18.218904 22699 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 10:21:18.220938 22699 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b4023f792f484136a49b2b8fd6accc05: Bootstrap starting.
I20260619 10:21:18.221432 22699 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b4023f792f484136a49b2b8fd6accc05: Neither blocks nor log segments found. Creating new log.
I20260619 10:21:18.221663 22699 log.cc:824] T 00000000000000000000000000000000 P b4023f792f484136a49b2b8fd6accc05: Log is configured to *not* fsync() on all Append() calls
I20260619 10:21:18.222268 22699 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b4023f792f484136a49b2b8fd6accc05: No bootstrap required, opened a new log
I20260619 10:21:18.223830 22699 raft_consensus.cc:348] T 00000000000000000000000000000000 P b4023f792f484136a49b2b8fd6accc05 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b4023f792f484136a49b2b8fd6accc05" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 40651 } }
I20260619 10:21:18.223979 22699 raft_consensus.cc:374] T 00000000000000000000000000000000 P b4023f792f484136a49b2b8fd6accc05 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 10:21:18.224017 22699 raft_consensus.cc:729] T 00000000000000000000000000000000 P b4023f792f484136a49b2b8fd6accc05 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b4023f792f484136a49b2b8fd6accc05, State: Initialized, Role: FOLLOWER
I20260619 10:21:18.223989 825 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTU9OP6/build/release/bin/kudu as pid 22631
I20260619 10:21:18.224123 825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781864224577058-825-0/minicluster-data/master-0/wal/instance
I20260619 10:21:18.224180 22699 consensus_queue.cc:260] T 00000000000000000000000000000000 P b4023f792f484136a49b2b8fd6accc05 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b4023f792f484136a49b2b8fd6accc05" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 40651 } }
I20260619 10:21:18.224267 22699 raft_consensus.cc:388] T 00000000000000000000000000000000 P b4023f792f484136a49b2b8fd6accc05 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 10:21:18.224309 22699 raft_consensus.cc:482] T 00000000000000000000000000000000 P b4023f792f484136a49b2b8fd6accc05 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 10:21:18.224337 825 external_mini_cluster.cc:1428] Setting key 590172b4e8a619528c9d01fb80689215
I20260619 10:21:18.224356 22699 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b4023f792f484136a49b2b8fd6accc05 [term 0 FOLLOWER]: Advancing to term 1
I20260619 10:21:18.225476 22699 raft_consensus.cc:504] T 00000000000000000000000000000000 P b4023f792f484136a49b2b8fd6accc05 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b4023f792f484136a49b2b8fd6accc05" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 40651 } }
I20260619 10:21:18.225580 22699 leader_election.cc:302] T 00000000000000000000000000000000 P b4023f792f484136a49b2b8fd6accc05 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b4023f792f484136a49b2b8fd6accc05; no voters:
I20260619 10:21:18.225733 22699 leader_election.cc:288] T 00000000000000000000000000000000 P b4023f792f484136a49b2b8fd6accc05 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 10:21:18.225757 22704 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b4023f792f484136a49b2b8fd6accc05 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 10:21:18.225859 22704 raft_consensus.cc:686] T 00000000000000000000000000000000 P b4023f792f484136a49b2b8fd6accc05 [term 1 LEADER]: Becoming Leader. State: Replica: b4023f792f484136a49b2b8fd6accc05, State: Running, Role: LEADER
I20260619 10:21:18.225973 22699 sys_catalog.cc:564] T 00000000000000000000000000000000 P b4023f792f484136a49b2b8fd6accc05 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 10:21:18.225996 22704 consensus_queue.cc:237] T 00000000000000000000000000000000 P b4023f792f484136a49b2b8fd6accc05 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b4023f792f484136a49b2b8fd6accc05" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 40651 } }
I20260619 10:21:18.226330 22705 sys_catalog.cc:455] T 00000000000000000000000000000000 P b4023f792f484136a49b2b8fd6accc05 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b4023f792f484136a49b2b8fd6accc05. Latest consensus state: current_term: 1 leader_uuid: "b4023f792f484136a49b2b8fd6accc05" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b4023f792f484136a49b2b8fd6accc05" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 40651 } } }
I20260619 10:21:18.226477 22705 sys_catalog.cc:458] T 00000000000000000000000000000000 P b4023f792f484136a49b2b8fd6accc05 [sys.catalog]: This master's current role is: LEADER
I20260619 10:21:18.227615 22712 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 10:21:18.227622 22706 sys_catalog.cc:455] T 00000000000000000000000000000000 P b4023f792f484136a49b2b8fd6accc05 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b4023f792f484136a49b2b8fd6accc05" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b4023f792f484136a49b2b8fd6accc05" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 40651 } } }
I20260619 10:21:18.228579 22706 sys_catalog.cc:458] T 00000000000000000000000000000000 P b4023f792f484136a49b2b8fd6accc05 [sys.catalog]: This master's current role is: LEADER
I20260619 10:21:18.229019 22712 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 10:21:18.230527 22712 catalog_manager.cc:1347] Generated new cluster ID: eb7a00509fa74d6dbb8908216ac6bc26
I20260619 10:21:18.230578 22712 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Jun 19 10:21:18 dist-test-slave-6pt2 krb5kdc[22618](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864478, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.206.126@KRBTEST.COM
I20260619 10:21:18.234853 22703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:21:18.224791 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43513 (local address 127.0.206.126:40651)
0619 10:21:18.225041 (+ 250us) server_negotiation.cc:206] Beginning negotiation
0619 10:21:18.225053 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0619 10:21:18.225090 (+ 37us) server_negotiation.cc:407] Connection header received
0619 10:21:18.225218 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:21:18.225224 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:21:18.225276 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:21:18.225438 (+ 162us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:21:18.226176 (+ 738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:21:18.227662 (+ 1486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:21:18.228343 (+ 681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:21:18.228906 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:21:18.229050 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:21:18.231518 (+ 2468us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 10:21:18.231522 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 10:21:18.231523 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 10:21:18.231533 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0619 10:21:18.233104 (+ 1571us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:21:18.233463 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:21:18.233466 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:21:18.233467 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:21:18.233556 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:21:18.233754 (+ 198us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:21:18.233755 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:21:18.233755 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:21:18.233967 (+ 212us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 10:21:18.234500 (+ 533us) server_negotiation.cc:1035] Waiting for connection context
0619 10:21:18.234551 (+ 51us) server_negotiation.cc:299] Negotiation successful
0619 10:21:18.234705 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":2,"server-negotiator.queue_time_us":198,"thread_start_us":134,"threads_started":1}
I20260619 10:21:18.235706 825 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 10:21:18.240711 22712 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 10:21:18.241333 22712 catalog_manager.cc:1504] Loading token signing keys...
I20260619 10:21:18.244570 22703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:21:18.236982 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37343 (local address 127.0.206.126:40651)
0619 10:21:18.237014 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0619 10:21:18.237016 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 10:21:18.238784 (+ 1768us) server_negotiation.cc:407] Connection header received
0619 10:21:18.238845 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:21:18.238846 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:21:18.238857 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:21:18.238912 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:21:18.239409 (+ 497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:21:18.239868 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:21:18.240607 (+ 739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:21:18.240880 (+ 273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:21:18.240952 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:21:18.242022 (+ 1070us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 10:21:18.242023 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 10:21:18.242023 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 10:21:18.242029 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 10:21:18.243505 (+ 1476us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:21:18.243849 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:21:18.243850 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:21:18.243850 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:21:18.243898 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:21:18.244094 (+ 196us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:21:18.244094 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:21:18.244095 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:21:18.244197 (+ 102us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 10:21:18.244302 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0619 10:21:18.244396 (+ 94us) server_negotiation.cc:299] Negotiation successful
0619 10:21:18.244484 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260619 10:21:18.249300 22712 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b4023f792f484136a49b2b8fd6accc05: Generated new TSK 0
I20260619 10:21:18.249444 22712 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 10:21:18.254128 825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTU9OP6/build/release/bin/kudu with pid 22631
2026-06-19T10:21:18Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (1274 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781864224577058-825-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 10:21:18 dist-test-slave-6pt2 krb5kdc[22730](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 10:21:18 dist-test-slave-6pt2 krb5kdc[22730](info): set up 2 sockets
Jun 19 10:21:18 dist-test-slave-6pt2 krb5kdc[22730](info): commencing operation
krb5kdc: starting...
W20260619 10:21:19.278872 825 mini_kdc.cc:121] Time spent starting KDC: real 1.007s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781864224577058-825-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781864224577058-825-0/krb5kdc/test-user.keytab.
Jun 19 10:21:19 dist-test-slave-6pt2 krb5kdc[22730](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864479, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T10:21:19Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T10:21:19Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.206.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.206.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.206.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781864224577058-825-0/krb5kdc/kudu_127.0.206.126.keytab.
Entry for principal kudu/127.0.206.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781864224577058-825-0/krb5kdc/kudu_127.0.206.126.keytab.
I20260619 10:21:19.386852 825 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTU9OP6/build/release/bin/kudu
/tmp/dist-test-taskTU9OP6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781864224577058-825-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781864224577058-825-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781864224577058-825-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781864224577058-825-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.206.126:43485
--webserver_interface=127.0.206.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781864224577058-825-0/krb5kdc/kudu_127.0.206.126.keytab
--principal=kudu/127.0.206.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.206.84:35153
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.206.126:43485
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781864224577058-825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781864224577058-825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781864224577058-825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 10:21:19.465126 22743 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 10:21:19.465301 22743 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 10:21:19.465323 22743 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 10:21:19.466398 22743 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 10:21:19.466439 22743 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 10:21:19.466452 22743 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 10:21:19.466464 22743 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 10:21:19.466476 22743 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 10:21:19.467823 22743 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.206.84:35153
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781864224577058-825-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781864224577058-825-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.206.126:43485
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781864224577058-825-0/krb5kdc/kudu_127.0.206.126.keytab
--principal=kudu/127.0.206.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.206.126:43485
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781864224577058-825-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.206.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22743
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781864224577058-825-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 09:32:44 UTC on ccycloud.quasar-gbtldp.root.comops.site
build id 4455440
I20260619 10:21:19.467990 22743 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 10:21:19.468150 22743 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 10:21:19.472469 22748 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 10:21:19.472570 22743 server_base.cc:1034] running on GCE node
W20260619 10:21:19.472486 22749 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 10:21:19.472486 22751 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 10:21:19.472872 22743 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 10:21:19.473132 22743 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 10:21:19.474267 22743 hybrid_clock.cc:648] HybridClock initialized: now 1781864479474252 us; error 37 us; skew 500 ppm
Jun 19 10:21:19 dist-test-slave-6pt2 krb5kdc[22730](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864479, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 10:21:19.481071 22743 init.cc:377] Logged in from keytab as kudu/127.0.206.126@KRBTEST.COM (short username kudu)
I20260619 10:21:19.481436 22743 webserver.cc:460] Webserver started at http://127.0.206.126:33049/ using document root <none> and password file <none>
I20260619 10:21:19.481626 22743 fs_manager.cc:359] Metadata directory not provided
I20260619 10:21:19.481665 22743 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 10:21:19.481750 22743 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 10:21:19.482766 22743 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781864224577058-825-0/minicluster-data/master-0/data/instance:
uuid: "cbbce3badcdc4eb9b628bfccb645f8b7"
format_stamp: "Formatted at 2026-06-19 10:21:19 on dist-test-slave-6pt2"
server_key: "394b06c7131e8ee2dd423259c61dcf70"
server_key_iv: "27d8be25120af484739dd1ab86fbd494"
server_key_version: "encryptionkey@0"
I20260619 10:21:19.483055 22743 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781864224577058-825-0/minicluster-data/master-0/wal/instance:
uuid: "cbbce3badcdc4eb9b628bfccb645f8b7"
format_stamp: "Formatted at 2026-06-19 10:21:19 on dist-test-slave-6pt2"
server_key: "394b06c7131e8ee2dd423259c61dcf70"
server_key_iv: "27d8be25120af484739dd1ab86fbd494"
server_key_version: "encryptionkey@0"
I20260619 10:21:19.484192 22743 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260619 10:21:19.484812 22758 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 10:21:19.484968 22743 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260619 10:21:19.485049 22743 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781864224577058-825-0/minicluster-data/master-0/data,/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781864224577058-825-0/minicluster-data/master-0/wal
uuid: "cbbce3badcdc4eb9b628bfccb645f8b7"
format_stamp: "Formatted at 2026-06-19 10:21:19 on dist-test-slave-6pt2"
server_key: "394b06c7131e8ee2dd423259c61dcf70"
server_key_iv: "27d8be25120af484739dd1ab86fbd494"
server_key_version: "encryptionkey@0"
I20260619 10:21:19.485097 22743 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781864224577058-825-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781864224577058-825-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781864224577058-825-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 10:21:19.512482 22743 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 10:21:19.514155 22743 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 10:21:19.514256 22743 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 10:21:19.517911 22743 rpc_server.cc:307] RPC server started. Bound to: 127.0.206.126:43485
I20260619 10:21:19.517967 22810 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.206.126:43485 every 8 connection(s)
I20260619 10:21:19.518286 22743 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781864224577058-825-0/minicluster-data/master-0/data/info.pb
I20260619 10:21:19.519140 22811 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 10:21:19.521000 825 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTU9OP6/build/release/bin/kudu as pid 22743
I20260619 10:21:19.521112 825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781864224577058-825-0/minicluster-data/master-0/wal/instance
I20260619 10:21:19.521307 825 external_mini_cluster.cc:1428] Setting key 13612ced3934a4c8f7681873ec37e55a
I20260619 10:21:19.521518 22811 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cbbce3badcdc4eb9b628bfccb645f8b7: Bootstrap starting.
I20260619 10:21:19.522179 22811 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cbbce3badcdc4eb9b628bfccb645f8b7: Neither blocks nor log segments found. Creating new log.
I20260619 10:21:19.522455 22811 log.cc:824] T 00000000000000000000000000000000 P cbbce3badcdc4eb9b628bfccb645f8b7: Log is configured to *not* fsync() on all Append() calls
I20260619 10:21:19.523164 22811 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cbbce3badcdc4eb9b628bfccb645f8b7: No bootstrap required, opened a new log
I20260619 10:21:19.525223 22811 raft_consensus.cc:348] T 00000000000000000000000000000000 P cbbce3badcdc4eb9b628bfccb645f8b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cbbce3badcdc4eb9b628bfccb645f8b7" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 43485 } }
I20260619 10:21:19.525377 22811 raft_consensus.cc:374] T 00000000000000000000000000000000 P cbbce3badcdc4eb9b628bfccb645f8b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 10:21:19.525413 22811 raft_consensus.cc:729] T 00000000000000000000000000000000 P cbbce3badcdc4eb9b628bfccb645f8b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cbbce3badcdc4eb9b628bfccb645f8b7, State: Initialized, Role: FOLLOWER
I20260619 10:21:19.525624 22811 consensus_queue.cc:260] T 00000000000000000000000000000000 P cbbce3badcdc4eb9b628bfccb645f8b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cbbce3badcdc4eb9b628bfccb645f8b7" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 43485 } }
I20260619 10:21:19.525738 22811 raft_consensus.cc:388] T 00000000000000000000000000000000 P cbbce3badcdc4eb9b628bfccb645f8b7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 10:21:19.525784 22811 raft_consensus.cc:482] T 00000000000000000000000000000000 P cbbce3badcdc4eb9b628bfccb645f8b7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 10:21:19.525853 22811 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cbbce3badcdc4eb9b628bfccb645f8b7 [term 0 FOLLOWER]: Advancing to term 1
I20260619 10:21:19.526345 22811 raft_consensus.cc:504] T 00000000000000000000000000000000 P cbbce3badcdc4eb9b628bfccb645f8b7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cbbce3badcdc4eb9b628bfccb645f8b7" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 43485 } }
I20260619 10:21:19.526455 22811 leader_election.cc:302] T 00000000000000000000000000000000 P cbbce3badcdc4eb9b628bfccb645f8b7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cbbce3badcdc4eb9b628bfccb645f8b7; no voters:
I20260619 10:21:19.526667 22811 leader_election.cc:288] T 00000000000000000000000000000000 P cbbce3badcdc4eb9b628bfccb645f8b7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 10:21:19.526710 22816 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cbbce3badcdc4eb9b628bfccb645f8b7 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 10:21:19.526826 22816 raft_consensus.cc:686] T 00000000000000000000000000000000 P cbbce3badcdc4eb9b628bfccb645f8b7 [term 1 LEADER]: Becoming Leader. State: Replica: cbbce3badcdc4eb9b628bfccb645f8b7, State: Running, Role: LEADER
I20260619 10:21:19.526880 22814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:21:19.521796 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50275 (local address 127.0.206.126:43485)
0619 10:21:19.521956 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0619 10:21:19.521965 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 10:21:19.521999 (+ 34us) server_negotiation.cc:407] Connection header received
0619 10:21:19.522459 (+ 460us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:21:19.522465 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:21:19.522532 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:21:19.522727 (+ 195us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:21:19.522886 (+ 159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:21:19.523793 (+ 907us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:21:19.524919 (+ 1126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:21:19.525289 (+ 370us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:21:19.525363 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:21:19.526119 (+ 756us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 10:21:19.526122 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 10:21:19.526123 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 10:21:19.526287 (+ 164us) server_negotiation.cc:922] calling sasl_server_start()
0619 10:21:19.526319 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 10:21:19.526329 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 10:21:19.526404 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0619 10:21:19.526490 (+ 86us) server_negotiation.cc:299] Negotiation successful
0619 10:21:19.526548 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":113,"thread_start_us":85,"threads_started":1}
I20260619 10:21:19.526942 22816 consensus_queue.cc:237] T 00000000000000000000000000000000 P cbbce3badcdc4eb9b628bfccb645f8b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cbbce3badcdc4eb9b628bfccb645f8b7" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 43485 } }
I20260619 10:21:19.526939 22811 sys_catalog.cc:564] T 00000000000000000000000000000000 P cbbce3badcdc4eb9b628bfccb645f8b7 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 10:21:19.527326 22817 sys_catalog.cc:455] T 00000000000000000000000000000000 P cbbce3badcdc4eb9b628bfccb645f8b7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "cbbce3badcdc4eb9b628bfccb645f8b7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cbbce3badcdc4eb9b628bfccb645f8b7" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 43485 } } }
I20260619 10:21:19.527335 22818 sys_catalog.cc:455] T 00000000000000000000000000000000 P cbbce3badcdc4eb9b628bfccb645f8b7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader cbbce3badcdc4eb9b628bfccb645f8b7. Latest consensus state: current_term: 1 leader_uuid: "cbbce3badcdc4eb9b628bfccb645f8b7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cbbce3badcdc4eb9b628bfccb645f8b7" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 43485 } } }
I20260619 10:21:19.527453 22817 sys_catalog.cc:458] T 00000000000000000000000000000000 P cbbce3badcdc4eb9b628bfccb645f8b7 [sys.catalog]: This master's current role is: LEADER
I20260619 10:21:19.527483 22818 sys_catalog.cc:458] T 00000000000000000000000000000000 P cbbce3badcdc4eb9b628bfccb645f8b7 [sys.catalog]: This master's current role is: LEADER
I20260619 10:21:19.527675 22823 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 10:21:19.528295 22823 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 10:21:19.528414 825 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 10:21:19.531481 22823 catalog_manager.cc:1347] Generated new cluster ID: cc638b79909f4073a387e58ee1c713e0
I20260619 10:21:19.531527 22823 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 10:21:19.532900 22814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:21:19.529628 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41551 (local address 127.0.206.126:43485)
0619 10:21:19.529676 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0619 10:21:19.529677 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 10:21:19.529826 (+ 149us) server_negotiation.cc:407] Connection header received
0619 10:21:19.529896 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:21:19.529897 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:21:19.529907 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:21:19.529962 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:21:19.530047 (+ 85us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:21:19.530743 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:21:19.531702 (+ 959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:21:19.531975 (+ 273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:21:19.532015 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:21:19.532669 (+ 654us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 10:21:19.532670 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 10:21:19.532671 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 10:21:19.532674 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0619 10:21:19.532686 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 10:21:19.532690 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 10:21:19.532715 (+ 25us) server_negotiation.cc:1035] Waiting for connection context
0619 10:21:19.532813 (+ 98us) server_negotiation.cc:299] Negotiation successful
0619 10:21:19.532831 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260619 10:21:19.546533 22823 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 10:21:19.547043 22823 catalog_manager.cc:1504] Loading token signing keys...
I20260619 10:21:19.556345 22823 catalog_manager.cc:5868] T 00000000000000000000000000000000 P cbbce3badcdc4eb9b628bfccb645f8b7: Generated new TSK 0
I20260619 10:21:19.556507 22823 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 10:21:19.560117 825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTU9OP6/build/release/bin/kudu with pid 22743
2026-06-19T10:21:19Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (1305 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781864224577058-825-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 10:21:19 dist-test-slave-6pt2 krb5kdc[22842](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 10:21:19 dist-test-slave-6pt2 krb5kdc[22842](info): set up 2 sockets
Jun 19 10:21:19 dist-test-slave-6pt2 krb5kdc[22842](info): commencing operation
krb5kdc: starting...
W20260619 10:21:20.562856 825 mini_kdc.cc:121] Time spent starting KDC: real 0.985s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781864224577058-825-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781864224577058-825-0/krb5kdc/test-user.keytab.
Jun 19 10:21:20 dist-test-slave-6pt2 krb5kdc[22842](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864480, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T10:21:20Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T10:21:20Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.206.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.206.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.206.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781864224577058-825-0/krb5kdc/kudu_127.0.206.126.keytab.
Entry for principal kudu/127.0.206.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781864224577058-825-0/krb5kdc/kudu_127.0.206.126.keytab.
I20260619 10:21:20.671255 825 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTU9OP6/build/release/bin/kudu
/tmp/dist-test-taskTU9OP6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781864224577058-825-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781864224577058-825-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781864224577058-825-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781864224577058-825-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.206.126:33535
--webserver_interface=127.0.206.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781864224577058-825-0/krb5kdc/kudu_127.0.206.126.keytab
--principal=kudu/127.0.206.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.206.84:39701
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.206.126:33535
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781864224577058-825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781864224577058-825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781864224577058-825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 10:21:20.748839 22855 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 10:21:20.749010 22855 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 10:21:20.749032 22855 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 10:21:20.750100 22855 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 10:21:20.750141 22855 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 10:21:20.750155 22855 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 10:21:20.750167 22855 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 10:21:20.750209 22855 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 10:21:20.751629 22855 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.206.84:39701
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781864224577058-825-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781864224577058-825-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.206.126:33535
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781864224577058-825-0/krb5kdc/kudu_127.0.206.126.keytab
--principal=kudu/127.0.206.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.206.126:33535
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781864224577058-825-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.206.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22855
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781864224577058-825-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 09:32:44 UTC on ccycloud.quasar-gbtldp.root.comops.site
build id 4455440
I20260619 10:21:20.751823 22855 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 10:21:20.752056 22855 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 10:21:20.756557 22860 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 10:21:20.756652 22863 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 10:21:20.756589 22861 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 10:21:20.756767 22855 server_base.cc:1034] running on GCE node
I20260619 10:21:20.757010 22855 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 10:21:20.757313 22855 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 10:21:20.758457 22855 hybrid_clock.cc:648] HybridClock initialized: now 1781864480758430 us; error 38 us; skew 500 ppm
Jun 19 10:21:20 dist-test-slave-6pt2 krb5kdc[22842](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864480, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 10:21:20.760795 22855 init.cc:377] Logged in from keytab as kudu/127.0.206.126@KRBTEST.COM (short username kudu)
I20260619 10:21:20.761256 22855 webserver.cc:460] Webserver started at http://127.0.206.126:42363/ using document root <none> and password file <none>
I20260619 10:21:20.761515 22855 fs_manager.cc:359] Metadata directory not provided
I20260619 10:21:20.761581 22855 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 10:21:20.761729 22855 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 10:21:20.762849 22855 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781864224577058-825-0/minicluster-data/master-0/data/instance:
uuid: "935347cf529446779a43aec7c5bf0436"
format_stamp: "Formatted at 2026-06-19 10:21:20 on dist-test-slave-6pt2"
server_key: "5c2e28d7ed9af3b3d5feb884511572cf"
server_key_iv: "2024a462223ee6823cddaedf2c8d4a8f"
server_key_version: "encryptionkey@0"
I20260619 10:21:20.763209 22855 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781864224577058-825-0/minicluster-data/master-0/wal/instance:
uuid: "935347cf529446779a43aec7c5bf0436"
format_stamp: "Formatted at 2026-06-19 10:21:20 on dist-test-slave-6pt2"
server_key: "5c2e28d7ed9af3b3d5feb884511572cf"
server_key_iv: "2024a462223ee6823cddaedf2c8d4a8f"
server_key_version: "encryptionkey@0"
I20260619 10:21:20.764498 22855 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 10:21:20.765182 22870 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 10:21:20.765342 22855 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 10:21:20.765436 22855 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781864224577058-825-0/minicluster-data/master-0/data,/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781864224577058-825-0/minicluster-data/master-0/wal
uuid: "935347cf529446779a43aec7c5bf0436"
format_stamp: "Formatted at 2026-06-19 10:21:20 on dist-test-slave-6pt2"
server_key: "5c2e28d7ed9af3b3d5feb884511572cf"
server_key_iv: "2024a462223ee6823cddaedf2c8d4a8f"
server_key_version: "encryptionkey@0"
I20260619 10:21:20.765544 22855 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781864224577058-825-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781864224577058-825-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781864224577058-825-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 10:21:20.768846 22855 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 10:21:20.770656 22855 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 10:21:20.770792 22855 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 10:21:20.774724 22855 rpc_server.cc:307] RPC server started. Bound to: 127.0.206.126:33535
I20260619 10:21:20.774760 22922 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.206.126:33535 every 8 connection(s)
I20260619 10:21:20.775070 22855 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781864224577058-825-0/minicluster-data/master-0/data/info.pb
I20260619 10:21:20.775735 825 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTU9OP6/build/release/bin/kudu as pid 22855
I20260619 10:21:20.775849 825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781864224577058-825-0/minicluster-data/master-0/wal/instance
I20260619 10:21:20.776041 825 external_mini_cluster.cc:1428] Setting key 760402fdc7b0d999ffd492ae7b3f58e5
I20260619 10:21:20.776064 22923 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 10:21:20.778402 22926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:21:20.776551 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44251 (local address 127.0.206.126:33535)
0619 10:21:20.776728 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0619 10:21:20.776737 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 10:21:20.776767 (+ 30us) server_negotiation.cc:407] Connection header received
0619 10:21:20.777181 (+ 414us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:21:20.777188 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:21:20.777220 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:21:20.777361 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:21:20.777994 (+ 633us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 10:21:20.777998 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 10:21:20.777998 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 10:21:20.778003 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0619 10:21:20.778054 (+ 51us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 10:21:20.778068 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 10:21:20.778139 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0619 10:21:20.778191 (+ 52us) server_negotiation.cc:299] Negotiation successful
0619 10:21:20.778228 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":126,"thread_start_us":76,"threads_started":1}
I20260619 10:21:20.779268 825 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 10:21:20.779547 22923 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 935347cf529446779a43aec7c5bf0436: Bootstrap starting.
I20260619 10:21:20.780081 22923 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 935347cf529446779a43aec7c5bf0436: Neither blocks nor log segments found. Creating new log.
I20260619 10:21:20.780400 22923 log.cc:824] T 00000000000000000000000000000000 P 935347cf529446779a43aec7c5bf0436: Log is configured to *not* fsync() on all Append() calls
I20260619 10:21:20.781307 22923 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 935347cf529446779a43aec7c5bf0436: No bootstrap required, opened a new log
I20260619 10:21:20.781590 22926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:21:20.780686 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58841 (local address 127.0.206.126:33535)
0619 10:21:20.780715 (+ 29us) server_negotiation.cc:206] Beginning negotiation
0619 10:21:20.780716 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 10:21:20.780722 (+ 6us) server_negotiation.cc:407] Connection header received
0619 10:21:20.780740 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:21:20.780740 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:21:20.780743 (+ 3us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:21:20.780786 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:21:20.781368 (+ 582us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 10:21:20.781370 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 10:21:20.781370 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 10:21:20.781374 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0619 10:21:20.781387 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 10:21:20.781391 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 10:21:20.781447 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0619 10:21:20.781541 (+ 94us) server_negotiation.cc:299] Negotiation successful
0619 10:21:20.781552 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260619 10:21:20.783334 22923 raft_consensus.cc:348] T 00000000000000000000000000000000 P 935347cf529446779a43aec7c5bf0436 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "935347cf529446779a43aec7c5bf0436" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 33535 } }
I20260619 10:21:20.783453 22923 raft_consensus.cc:374] T 00000000000000000000000000000000 P 935347cf529446779a43aec7c5bf0436 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 10:21:20.783474 22923 raft_consensus.cc:729] T 00000000000000000000000000000000 P 935347cf529446779a43aec7c5bf0436 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 935347cf529446779a43aec7c5bf0436, State: Initialized, Role: FOLLOWER
I20260619 10:21:20.783622 22923 consensus_queue.cc:260] T 00000000000000000000000000000000 P 935347cf529446779a43aec7c5bf0436 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "935347cf529446779a43aec7c5bf0436" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 33535 } }
I20260619 10:21:20.783697 22923 raft_consensus.cc:388] T 00000000000000000000000000000000 P 935347cf529446779a43aec7c5bf0436 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 10:21:20.783756 22923 raft_consensus.cc:482] T 00000000000000000000000000000000 P 935347cf529446779a43aec7c5bf0436 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 10:21:20.783806 22923 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 935347cf529446779a43aec7c5bf0436 [term 0 FOLLOWER]: Advancing to term 1
I20260619 10:21:20.784402 22923 raft_consensus.cc:504] T 00000000000000000000000000000000 P 935347cf529446779a43aec7c5bf0436 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "935347cf529446779a43aec7c5bf0436" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 33535 } }
I20260619 10:21:20.784507 22923 leader_election.cc:302] T 00000000000000000000000000000000 P 935347cf529446779a43aec7c5bf0436 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 935347cf529446779a43aec7c5bf0436; no voters:
I20260619 10:21:20.784665 22923 leader_election.cc:288] T 00000000000000000000000000000000 P 935347cf529446779a43aec7c5bf0436 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 10:21:20.784687 22933 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 935347cf529446779a43aec7c5bf0436 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 10:21:20.784819 22933 raft_consensus.cc:686] T 00000000000000000000000000000000 P 935347cf529446779a43aec7c5bf0436 [term 1 LEADER]: Becoming Leader. State: Replica: 935347cf529446779a43aec7c5bf0436, State: Running, Role: LEADER
I20260619 10:21:20.784894 22923 sys_catalog.cc:564] T 00000000000000000000000000000000 P 935347cf529446779a43aec7c5bf0436 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 10:21:20.784936 22933 consensus_queue.cc:237] T 00000000000000000000000000000000 P 935347cf529446779a43aec7c5bf0436 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "935347cf529446779a43aec7c5bf0436" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 33535 } }
I20260619 10:21:20.785329 22934 sys_catalog.cc:455] T 00000000000000000000000000000000 P 935347cf529446779a43aec7c5bf0436 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "935347cf529446779a43aec7c5bf0436" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "935347cf529446779a43aec7c5bf0436" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 33535 } } }
I20260619 10:21:20.785332 22935 sys_catalog.cc:455] T 00000000000000000000000000000000 P 935347cf529446779a43aec7c5bf0436 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 935347cf529446779a43aec7c5bf0436. Latest consensus state: current_term: 1 leader_uuid: "935347cf529446779a43aec7c5bf0436" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "935347cf529446779a43aec7c5bf0436" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 33535 } } }
I20260619 10:21:20.785471 22934 sys_catalog.cc:458] T 00000000000000000000000000000000 P 935347cf529446779a43aec7c5bf0436 [sys.catalog]: This master's current role is: LEADER
I20260619 10:21:20.785547 22935 sys_catalog.cc:458] T 00000000000000000000000000000000 P 935347cf529446779a43aec7c5bf0436 [sys.catalog]: This master's current role is: LEADER
I20260619 10:21:20.785737 22942 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 10:21:20.786243 22942 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 10:21:20.787750 22942 catalog_manager.cc:1347] Generated new cluster ID: 25c9aa52d99644c99d1fa7e845a322bd
I20260619 10:21:20.787798 22942 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 10:21:20.797642 22942 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 10:21:20.797782 22942 catalog_manager.cc:1504] Loading token signing keys...
I20260619 10:21:20.811281 22942 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 935347cf529446779a43aec7c5bf0436: Generated new TSK 0
I20260619 10:21:20.811429 22942 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 10:21:20.815299 825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTU9OP6/build/release/bin/kudu with pid 22855
2026-06-19T10:21:20Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (1255 ms)
[----------] 18 tests from AuthTokenIssuingTest (22936 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781864224577058-825-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 10:21:20 dist-test-slave-6pt2 krb5kdc[22954](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 10:21:20 dist-test-slave-6pt2 krb5kdc[22954](info): set up 2 sockets
Jun 19 10:21:20 dist-test-slave-6pt2 krb5kdc[22954](info): commencing operation
krb5kdc: starting...
W20260619 10:21:21.838725 825 mini_kdc.cc:121] Time spent starting KDC: real 1.006s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781864224577058-825-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781864224577058-825-0/krb5kdc/test-user.keytab.
Jun 19 10:21:21 dist-test-slave-6pt2 krb5kdc[22954](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864481, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T10:21:21Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T10:21:21Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.206.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.206.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.206.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781864224577058-825-0/krb5kdc/kudu_127.0.206.126.keytab.
Entry for principal kudu/127.0.206.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781864224577058-825-0/krb5kdc/kudu_127.0.206.126.keytab.
I20260619 10:21:21.947572 825 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTU9OP6/build/release/bin/kudu
/tmp/dist-test-taskTU9OP6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781864224577058-825-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781864224577058-825-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781864224577058-825-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781864224577058-825-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.206.126:35571
--webserver_interface=127.0.206.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781864224577058-825-0/krb5kdc/kudu_127.0.206.126.keytab
--principal=kudu/127.0.206.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.206.84:37443
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.206.126:35571,127.0.206.125:43989,127.0.206.124:45531
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781864224577058-825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781864224577058-825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781864224577058-825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 10:21:22.025660 22967 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260619 10:21:22.025908 22967 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 10:21:22.025938 22967 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 10:21:22.025980 22967 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 10:21:22.027086 22967 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 10:21:22.027151 22967 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 10:21:22.027173 22967 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 10:21:22.027194 22967 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 10:21:22.027221 22967 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 10:21:22.028589 22967 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.206.84:37443
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781864224577058-825-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781864224577058-825-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.206.126:35571,127.0.206.125:43989,127.0.206.124:45531
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781864224577058-825-0/krb5kdc/kudu_127.0.206.126.keytab
--principal=kudu/127.0.206.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.206.126:35571
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781864224577058-825-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.206.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22967
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781864224577058-825-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 09:32:44 UTC on ccycloud.quasar-gbtldp.root.comops.site
build id 4455440
I20260619 10:21:22.028846 22967 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 10:21:22.029057 22967 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 10:21:22.033749 22973 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 10:21:22.033854 22967 server_base.cc:1034] running on GCE node
W20260619 10:21:22.033768 22975 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 10:21:22.033764 22972 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 10:21:22.034194 22967 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 10:21:22.034497 22967 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 10:21:22.035661 22967 hybrid_clock.cc:648] HybridClock initialized: now 1781864482035654 us; error 43 us; skew 500 ppm
Jun 19 10:21:22 dist-test-slave-6pt2 krb5kdc[22954](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864482, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 10:21:22.037962 22967 init.cc:377] Logged in from keytab as kudu/127.0.206.126@KRBTEST.COM (short username kudu)
I20260619 10:21:22.038352 22967 webserver.cc:460] Webserver started at http://127.0.206.126:38621/ using document root <none> and password file <none>
I20260619 10:21:22.038599 22967 fs_manager.cc:359] Metadata directory not provided
I20260619 10:21:22.038710 22967 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 10:21:22.038858 22967 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 10:21:22.039917 22967 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781864224577058-825-0/minicluster-data/master-0/data/instance:
uuid: "9e41ae07a1254bf38eadf9d3dd92205f"
format_stamp: "Formatted at 2026-06-19 10:21:22 on dist-test-slave-6pt2"
server_key: "894234fd89123696bf648b0cb7155bb0"
server_key_iv: "26df628a8495fe2f0860c5e045fd2a96"
server_key_version: "encryptionkey@0"
I20260619 10:21:22.040278 22967 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781864224577058-825-0/minicluster-data/master-0/wal/instance:
uuid: "9e41ae07a1254bf38eadf9d3dd92205f"
format_stamp: "Formatted at 2026-06-19 10:21:22 on dist-test-slave-6pt2"
server_key: "894234fd89123696bf648b0cb7155bb0"
server_key_iv: "26df628a8495fe2f0860c5e045fd2a96"
server_key_version: "encryptionkey@0"
I20260619 10:21:22.041569 22967 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 10:21:22.042279 22982 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 10:21:22.042441 22967 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 10:21:22.042560 22967 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781864224577058-825-0/minicluster-data/master-0/data,/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781864224577058-825-0/minicluster-data/master-0/wal
uuid: "9e41ae07a1254bf38eadf9d3dd92205f"
format_stamp: "Formatted at 2026-06-19 10:21:22 on dist-test-slave-6pt2"
server_key: "894234fd89123696bf648b0cb7155bb0"
server_key_iv: "26df628a8495fe2f0860c5e045fd2a96"
server_key_version: "encryptionkey@0"
I20260619 10:21:22.042665 22967 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781864224577058-825-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781864224577058-825-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781864224577058-825-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 10:21:22.060595 22967 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 10:21:22.062314 22967 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 10:21:22.062480 22967 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 10:21:22.066468 22967 rpc_server.cc:307] RPC server started. Bound to: 127.0.206.126:35571
I20260619 10:21:22.066512 23034 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.206.126:35571 every 8 connection(s)
I20260619 10:21:22.066877 22967 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781864224577058-825-0/minicluster-data/master-0/data/info.pb
I20260619 10:21:22.067836 23035 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 10:21:22.069358 23035 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.206.126" port: 35571 } has no permanent_uuid. Determining permanent_uuid...
I20260619 10:21:22.071578 825 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTU9OP6/build/release/bin/kudu as pid 22967
I20260619 10:21:22.071686 825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781864224577058-825-0/minicluster-data/master-0/wal/instance
I20260619 10:21:22.071873 825 external_mini_cluster.cc:1428] Setting key a3681ed7a3381cbc954ea1269d3f719a
Jun 19 10:21:22 dist-test-slave-6pt2 krb5kdc[22954](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864482, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.126@KRBTEST.COM for kudu/127.0.206.126@KRBTEST.COM
I20260619 10:21:22.079538 23037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:21:22.069936 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:35571 (local address 127.0.0.1:47182)
0619 10:21:22.070089 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0619 10:21:22.070129 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0619 10:21:22.070554 (+ 425us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 10:21:22.070833 (+ 279us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 10:21:22.070835 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 10:21:22.071060 (+ 225us) client_negotiation.cc:189] Negotiated authn=SASL
0619 10:21:22.071224 (+ 164us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:21:22.071228 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:21:22.072329 (+ 1101us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:21:22.072331 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:21:22.073403 (+ 1072us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:21:22.073406 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:21:22.074091 (+ 685us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:21:22.074092 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:21:22.074142 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:21:22.074681 (+ 539us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 10:21:22.074689 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 10:21:22.075972 (+ 1283us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 10:21:22.078056 (+ 2084us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 10:21:22.078062 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 10:21:22.078063 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 10:21:22.078254 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 10:21:22.078529 (+ 275us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 10:21:22.078530 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 10:21:22.078530 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 10:21:22.078668 (+ 138us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 10:21:22.079099 (+ 431us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 10:21:22.079104 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 10:21:22.079158 (+ 54us) client_negotiation.cc:769] Sending connection context
0619 10:21:22.079224 (+ 66us) client_negotiation.cc:240] Negotiation successful
0619 10:21:22.079343 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":91,"threads_started":1}
I20260619 10:21:22.079545 23038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:21:22.069993 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47182 (local address 127.0.206.126:35571)
0619 10:21:22.070137 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0619 10:21:22.070145 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 10:21:22.070217 (+ 72us) server_negotiation.cc:407] Connection header received
0619 10:21:22.070721 (+ 504us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:21:22.070723 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:21:22.070738 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:21:22.070812 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:21:22.071316 (+ 504us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:21:22.072239 (+ 923us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:21:22.073487 (+ 1248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:21:22.073985 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:21:22.074104 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:21:22.076056 (+ 1952us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 10:21:22.076065 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 10:21:22.076065 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 10:21:22.076073 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 10:21:22.077968 (+ 1895us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:21:22.078340 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:21:22.078343 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:21:22.078344 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:21:22.078448 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:21:22.078740 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:21:22.078741 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:21:22.078741 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:21:22.079024 (+ 283us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 10:21:22.079115 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0619 10:21:22.079320 (+ 205us) server_negotiation.cc:299] Negotiation successful
0619 10:21:22.079404 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":112,"thread_start_us":82,"threads_started":1}
WARNING: no policy specified for kudu/127.0.206.125@KRBTEST.COM; defaulting to no policy
I20260619 10:21:22.080277 23035 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.206.125" port: 43989 } has no permanent_uuid. Determining permanent_uuid...
I20260619 10:21:22.080622 23037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:21:22.080522 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.125:43989 (local address 127.0.0.1:39996)
0619 10:21:22.080566 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0619 10:21:22.080605 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.125:43989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260619 10:21:22.080825 22984 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.206.125:43989: connect: Connection refused (error 111)
I20260619 10:21:22.081190 23037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:21:22.081121 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.125:43989 (local address 127.0.0.1:40008)
0619 10:21:22.081141 (+ 20us) negotiation.cc:107] Waiting for socket to connect
0619 10:21:22.081176 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.125:43989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":12}
W20260619 10:21:22.081274 23035 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.206.125:43989: Network error: Client connection negotiation failed: client connection to 127.0.206.125:43989: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.206.125@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.206.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781864224577058-825-0/krb5kdc/kudu_127.0.206.125.keytab.
Entry for principal kudu/127.0.206.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781864224577058-825-0/krb5kdc/kudu_127.0.206.125.keytab.
I20260619 10:21:22.095574 825 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTU9OP6/build/release/bin/kudu
/tmp/dist-test-taskTU9OP6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781864224577058-825-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781864224577058-825-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781864224577058-825-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781864224577058-825-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.206.125:43989
--webserver_interface=127.0.206.125
--webserver_port=0
--keytab_file=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781864224577058-825-0/krb5kdc/kudu_127.0.206.125.keytab
--principal=kudu/127.0.206.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.206.84:37443
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.206.126:35571,127.0.206.125:43989,127.0.206.124:45531
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781864224577058-825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781864224577058-825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781864224577058-825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 10:21:22.130483 23035 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.206.125" port: 43989 } attempt: 1
I20260619 10:21:22.130975 23037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:21:22.130849 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.125:43989 (local address 127.0.0.1:40018)
0619 10:21:22.130905 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0619 10:21:22.130944 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.125:43989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
I20260619 10:21:22.131357 23037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:21:22.131261 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.125:43989 (local address 127.0.0.1:40030)
0619 10:21:22.131309 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0619 10:21:22.131335 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.125:43989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260619 10:21:22.131459 23035 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.206.125:43989: Network error: Client connection negotiation failed: client connection to 127.0.206.125:43989: connect: Connection refused (error 111)
W20260619 10:21:22.174111 23042 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260619 10:21:22.174341 23042 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 10:21:22.174383 23042 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 10:21:22.174422 23042 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 10:21:22.175572 23042 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 10:21:22.175633 23042 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 10:21:22.175657 23042 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 10:21:22.175684 23042 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 10:21:22.175714 23042 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 10:21:22.177161 23042 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.206.84:37443
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781864224577058-825-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781864224577058-825-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.0.206.126:35571,127.0.206.125:43989,127.0.206.124:45531
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781864224577058-825-0/krb5kdc/kudu_127.0.206.125.keytab
--principal=kudu/127.0.206.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.206.125:43989
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781864224577058-825-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.0.206.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23042
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781864224577058-825-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 09:32:44 UTC on ccycloud.quasar-gbtldp.root.comops.site
build id 4455440
I20260619 10:21:22.177440 23042 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 10:21:22.177659 23042 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260619 10:21:22.182281 23042 server_base.cc:1034] running on GCE node
W20260619 10:21:22.182265 23048 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 10:21:22.182253 23047 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 10:21:22.182260 23050 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 10:21:22.182813 23042 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 10:21:22.183049 23042 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 10:21:22.184193 23042 hybrid_clock.cc:648] HybridClock initialized: now 1781864482184181 us; error 28 us; skew 500 ppm
Jun 19 10:21:22 dist-test-slave-6pt2 krb5kdc[22954](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864482, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 10:21:22.186431 23042 init.cc:377] Logged in from keytab as kudu/127.0.206.125@KRBTEST.COM (short username kudu)
I20260619 10:21:22.186882 23042 webserver.cc:460] Webserver started at http://127.0.206.125:42159/ using document root <none> and password file <none>
I20260619 10:21:22.187104 23042 fs_manager.cc:359] Metadata directory not provided
I20260619 10:21:22.187145 23042 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 10:21:22.187276 23042 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 10:21:22.188280 23042 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781864224577058-825-0/minicluster-data/master-1/data/instance:
uuid: "eab431b395514e4492e49d1d11a90a54"
format_stamp: "Formatted at 2026-06-19 10:21:22 on dist-test-slave-6pt2"
server_key: "46e08ffe151b1e9c303b666557977a8b"
server_key_iv: "c2576d0c051449bf0680020bc3d70154"
server_key_version: "encryptionkey@0"
I20260619 10:21:22.188607 23042 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781864224577058-825-0/minicluster-data/master-1/wal/instance:
uuid: "eab431b395514e4492e49d1d11a90a54"
format_stamp: "Formatted at 2026-06-19 10:21:22 on dist-test-slave-6pt2"
server_key: "46e08ffe151b1e9c303b666557977a8b"
server_key_iv: "c2576d0c051449bf0680020bc3d70154"
server_key_version: "encryptionkey@0"
I20260619 10:21:22.189980 23042 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260619 10:21:22.190699 23057 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 10:21:22.190857 23042 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260619 10:21:22.190963 23042 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781864224577058-825-0/minicluster-data/master-1/data,/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781864224577058-825-0/minicluster-data/master-1/wal
uuid: "eab431b395514e4492e49d1d11a90a54"
format_stamp: "Formatted at 2026-06-19 10:21:22 on dist-test-slave-6pt2"
server_key: "46e08ffe151b1e9c303b666557977a8b"
server_key_iv: "c2576d0c051449bf0680020bc3d70154"
server_key_version: "encryptionkey@0"
I20260619 10:21:22.191039 23042 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781864224577058-825-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781864224577058-825-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781864224577058-825-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 10:21:22.199606 23035 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.206.125" port: 43989 } attempt: 2
I20260619 10:21:22.199965 23037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:21:22.199850 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.125:43989 (local address 127.0.0.1:40038)
0619 10:21:22.199900 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0619 10:21:22.199934 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.125:43989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
I20260619 10:21:22.200284 23037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:21:22.200197 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.125:43989 (local address 127.0.0.1:40048)
0619 10:21:22.200240 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0619 10:21:22.200269 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.125:43989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260619 10:21:22.200368 23035 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.206.125:43989: Network error: Client connection negotiation failed: client connection to 127.0.206.125:43989: connect: Connection refused (error 111)
I20260619 10:21:22.214823 23042 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 10:21:22.216500 23042 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 10:21:22.216655 23042 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 10:21:22.220728 23042 rpc_server.cc:307] RPC server started. Bound to: 127.0.206.125:43989
I20260619 10:21:22.220775 23109 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.206.125:43989 every 8 connection(s)
I20260619 10:21:22.221107 23042 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781864224577058-825-0/minicluster-data/master-1/data/info.pb
I20260619 10:21:22.222005 23110 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 10:21:22.223656 23110 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.206.126" port: 35571 } has no permanent_uuid. Determining permanent_uuid...
Jun 19 10:21:22 dist-test-slave-6pt2 krb5kdc[22954](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864482, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.125@KRBTEST.COM for kudu/127.0.206.126@KRBTEST.COM
I20260619 10:21:22.230026 825 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTU9OP6/build/release/bin/kudu as pid 23042
I20260619 10:21:22.230130 825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781864224577058-825-0/minicluster-data/master-1/wal/instance
I20260619 10:21:22.230283 825 external_mini_cluster.cc:1428] Setting key 6ccaa5d43f3134b61a114c4f7dbd50a1
I20260619 10:21:22.232283 23112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:21:22.224365 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:35571 (local address 127.0.0.1:47192)
0619 10:21:22.224532 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0619 10:21:22.224566 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0619 10:21:22.225007 (+ 441us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 10:21:22.225211 (+ 204us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 10:21:22.225214 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 10:21:22.225447 (+ 233us) client_negotiation.cc:189] Negotiated authn=SASL
0619 10:21:22.225623 (+ 176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:21:22.225627 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:21:22.226272 (+ 645us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:21:22.226273 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:21:22.227088 (+ 815us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:21:22.227090 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:21:22.227457 (+ 367us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:21:22.227458 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:21:22.227552 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:21:22.227959 (+ 407us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 10:21:22.227967 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 10:21:22.229209 (+ 1242us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 10:21:22.230841 (+ 1632us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 10:21:22.230847 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 10:21:22.230849 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 10:21:22.231165 (+ 316us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 10:21:22.231354 (+ 189us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 10:21:22.231355 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 10:21:22.231356 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 10:21:22.231485 (+ 129us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 10:21:22.231798 (+ 313us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 10:21:22.231803 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 10:21:22.231909 (+ 106us) client_negotiation.cc:769] Sending connection context
0619 10:21:22.232029 (+ 120us) client_negotiation.cc:240] Negotiation successful
0619 10:21:22.232144 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":84,"threads_started":1}
I20260619 10:21:22.232292 23038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:21:22.224376 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47192 (local address 127.0.206.126:35571)
0619 10:21:22.224425 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0619 10:21:22.224428 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 10:21:22.224646 (+ 218us) server_negotiation.cc:407] Connection header received
0619 10:21:22.225086 (+ 440us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:21:22.225088 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:21:22.225099 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:21:22.225161 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:21:22.225722 (+ 561us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:21:22.226195 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:21:22.227159 (+ 964us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:21:22.227367 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:21:22.227436 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:21:22.229289 (+ 1853us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 10:21:22.229290 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 10:21:22.229290 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 10:21:22.229297 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 10:21:22.230758 (+ 1461us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:21:22.231236 (+ 478us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:21:22.231237 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:21:22.231237 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:21:22.231291 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:21:22.231547 (+ 256us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:21:22.231548 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:21:22.231548 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:21:22.231723 (+ 175us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 10:21:22.231803 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0619 10:21:22.232074 (+ 271us) server_negotiation.cc:299] Negotiation successful
0619 10:21:22.232170 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260619 10:21:22.233039 23110 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.206.125" port: 43989 } has no permanent_uuid. Determining permanent_uuid...
Jun 19 10:21:22 dist-test-slave-6pt2 krb5kdc[22954](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864482, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.125@KRBTEST.COM for kudu/127.0.206.125@KRBTEST.COM
WARNING: no policy specified for kudu/127.0.206.124@KRBTEST.COM; defaulting to no policy
I20260619 10:21:22.240285 23112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:21:22.233284 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.125:43989 (local address 127.0.0.1:40056)
0619 10:21:22.233347 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0619 10:21:22.233357 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 10:21:22.233396 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 10:21:22.233668 (+ 272us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 10:21:22.233669 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 10:21:22.233796 (+ 127us) client_negotiation.cc:189] Negotiated authn=SASL
0619 10:21:22.233834 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:21:22.233835 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:21:22.234776 (+ 941us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:21:22.234777 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:21:22.235294 (+ 517us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:21:22.235296 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:21:22.235767 (+ 471us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:21:22.235768 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:21:22.235828 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:21:22.236213 (+ 385us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 10:21:22.236217 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 10:21:22.237684 (+ 1467us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 10:21:22.239490 (+ 1806us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 10:21:22.239491 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 10:21:22.239491 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 10:21:22.239652 (+ 161us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 10:21:22.239829 (+ 177us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 10:21:22.239830 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 10:21:22.239830 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 10:21:22.239876 (+ 46us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 10:21:22.240083 (+ 207us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 10:21:22.240084 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 10:21:22.240116 (+ 32us) client_negotiation.cc:769] Sending connection context
0619 10:21:22.240163 (+ 47us) client_negotiation.cc:240] Negotiation successful
0619 10:21:22.240230 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":48}
I20260619 10:21:22.240490 23114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:21:22.233385 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40056 (local address 127.0.206.125:43989)
0619 10:21:22.233533 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0619 10:21:22.233537 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 10:21:22.233560 (+ 23us) server_negotiation.cc:407] Connection header received
0619 10:21:22.233575 (+ 15us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:21:22.233577 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:21:22.233586 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:21:22.233667 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:21:22.233922 (+ 255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:21:22.234672 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:21:22.235388 (+ 716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:21:22.235681 (+ 293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:21:22.235767 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:21:22.237773 (+ 2006us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 10:21:22.237779 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 10:21:22.237780 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 10:21:22.237785 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0619 10:21:22.239352 (+ 1567us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:21:22.239715 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:21:22.239717 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:21:22.239717 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:21:22.239764 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:21:22.239917 (+ 153us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:21:22.239917 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:21:22.239917 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:21:22.240021 (+ 104us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 10:21:22.240087 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0619 10:21:22.240182 (+ 95us) server_negotiation.cc:299] Negotiation successful
0619 10:21:22.240279 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":36,"server-negotiator.queue_time_us":122,"thread_start_us":101,"threads_started":1}
I20260619 10:21:22.241047 23110 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.206.124" port: 45531 } has no permanent_uuid. Determining permanent_uuid...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.206.124@KRBTEST.COM" created.
I20260619 10:21:22.241330 23112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:21:22.241232 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.124:45531 (local address 127.0.0.1:55848)
0619 10:21:22.241277 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0619 10:21:22.241313 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.124:45531: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260619 10:21:22.241469 23060 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.206.124:45531: connect: Connection refused (error 111)
I20260619 10:21:22.241824 23112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:21:22.241745 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.124:45531 (local address 127.0.0.1:55852)
0619 10:21:22.241789 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0619 10:21:22.241811 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.124:45531: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260619 10:21:22.241966 23110 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.206.124:45531: Network error: Client connection negotiation failed: client connection to 127.0.206.124:45531: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.206.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781864224577058-825-0/krb5kdc/kudu_127.0.206.124.keytab.
Entry for principal kudu/127.0.206.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781864224577058-825-0/krb5kdc/kudu_127.0.206.124.keytab.
I20260619 10:21:22.255481 825 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTU9OP6/build/release/bin/kudu
/tmp/dist-test-taskTU9OP6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781864224577058-825-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781864224577058-825-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781864224577058-825-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781864224577058-825-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.206.124:45531
--webserver_interface=127.0.206.124
--webserver_port=0
--keytab_file=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781864224577058-825-0/krb5kdc/kudu_127.0.206.124.keytab
--principal=kudu/127.0.206.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.206.84:37443
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.206.126:35571,127.0.206.125:43989,127.0.206.124:45531
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781864224577058-825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781864224577058-825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781864224577058-825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 10:21:22.291183 23110 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.206.124" port: 45531 } attempt: 1
I20260619 10:21:22.291527 23035 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.206.125" port: 43989 } attempt: 3
I20260619 10:21:22.291706 23112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:21:22.291558 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.124:45531 (local address 127.0.0.1:55854)
0619 10:21:22.291625 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0619 10:21:22.291668 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.124:45531: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
I20260619 10:21:22.292042 23112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:21:22.291971 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.124:45531 (local address 127.0.0.1:55862)
0619 10:21:22.291997 (+ 26us) negotiation.cc:107] Waiting for socket to connect
0619 10:21:22.292023 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.124:45531: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":14}
W20260619 10:21:22.292131 23110 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.206.124:45531: Network error: Client connection negotiation failed: client connection to 127.0.206.124:45531: connect: Connection refused (error 111)
Jun 19 10:21:22 dist-test-slave-6pt2 krb5kdc[22954](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864482, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.126@KRBTEST.COM for kudu/127.0.206.125@KRBTEST.COM
I20260619 10:21:22.298398 23037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:21:22.291810 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.125:43989 (local address 127.0.0.1:40058)
0619 10:21:22.291848 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0619 10:21:22.291860 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 10:21:22.291919 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 10:21:22.292064 (+ 145us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 10:21:22.292065 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 10:21:22.292256 (+ 191us) client_negotiation.cc:189] Negotiated authn=SASL
0619 10:21:22.292317 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:21:22.292319 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:21:22.292944 (+ 625us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:21:22.292945 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:21:22.293630 (+ 685us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:21:22.293634 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:21:22.294120 (+ 486us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:21:22.294121 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:21:22.294212 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:21:22.294729 (+ 517us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 10:21:22.294735 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0619 10:21:22.295870 (+ 1135us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 10:21:22.297541 (+ 1671us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 10:21:22.297542 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 10:21:22.297542 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 10:21:22.297700 (+ 158us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 10:21:22.297889 (+ 189us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 10:21:22.297889 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 10:21:22.297889 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 10:21:22.297937 (+ 48us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 10:21:22.298184 (+ 247us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 10:21:22.298184 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 10:21:22.298219 (+ 35us) client_negotiation.cc:769] Sending connection context
0619 10:21:22.298269 (+ 50us) client_negotiation.cc:240] Negotiation successful
0619 10:21:22.298337 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":17}
I20260619 10:21:22.298441 23114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:21:22.291847 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40058 (local address 127.0.206.125:43989)
0619 10:21:22.291934 (+ 87us) server_negotiation.cc:206] Beginning negotiation
0619 10:21:22.291936 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 10:21:22.291946 (+ 10us) server_negotiation.cc:407] Connection header received
0619 10:21:22.291973 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:21:22.291973 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:21:22.291983 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:21:22.292055 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:21:22.292400 (+ 345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:21:22.292867 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:21:22.293716 (+ 849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:21:22.294047 (+ 331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:21:22.294117 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:21:22.295946 (+ 1829us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 10:21:22.295947 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 10:21:22.295947 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 10:21:22.295960 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0619 10:21:22.297461 (+ 1501us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:21:22.297768 (+ 307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:21:22.297769 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:21:22.297770 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:21:22.297827 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:21:22.297993 (+ 166us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:21:22.297994 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:21:22.297994 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:21:22.298122 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 10:21:22.298189 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0619 10:21:22.298291 (+ 102us) server_negotiation.cc:299] Negotiation successful
0619 10:21:22.298364 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260619 10:21:22.299119 23035 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.206.124" port: 45531 } has no permanent_uuid. Determining permanent_uuid...
I20260619 10:21:22.299412 23037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:21:22.299316 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.124:45531 (local address 127.0.0.1:55866)
0619 10:21:22.299360 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0619 10:21:22.299386 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.124:45531: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
I20260619 10:21:22.299742 23037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:21:22.299634 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.124:45531 (local address 127.0.0.1:55868)
0619 10:21:22.299675 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0619 10:21:22.299715 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.124:45531: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260619 10:21:22.299822 23035 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.206.124:45531: Network error: Client connection negotiation failed: client connection to 127.0.206.124:45531: connect: Connection refused (error 111)
I20260619 10:21:22.330991 23035 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.206.124" port: 45531 } attempt: 1
I20260619 10:21:22.331383 23037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:21:22.331254 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.124:45531 (local address 127.0.0.1:55870)
0619 10:21:22.331319 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0619 10:21:22.331354 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.124:45531: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
I20260619 10:21:22.331717 23037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:21:22.331622 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.124:45531 (local address 127.0.0.1:55878)
0619 10:21:22.331669 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0619 10:21:22.331696 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.124:45531: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260619 10:21:22.331805 23035 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.206.124:45531: Network error: Client connection negotiation failed: client connection to 127.0.206.124:45531: connect: Connection refused (error 111)
W20260619 10:21:22.335242 23117 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260619 10:21:22.335445 23117 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 10:21:22.335461 23117 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 10:21:22.335518 23117 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 10:21:22.336609 23117 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 10:21:22.336647 23117 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 10:21:22.336661 23117 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 10:21:22.336673 23117 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 10:21:22.336685 23117 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 10:21:22.338133 23117 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.206.84:37443
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781864224577058-825-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781864224577058-825-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.0.206.126:35571,127.0.206.125:43989,127.0.206.124:45531
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781864224577058-825-0/krb5kdc/kudu_127.0.206.124.keytab
--principal=kudu/127.0.206.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.206.124:45531
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781864224577058-825-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.0.206.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23117
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781864224577058-825-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 09:32:44 UTC on ccycloud.quasar-gbtldp.root.comops.site
build id 4455440
I20260619 10:21:22.338362 23117 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 10:21:22.338580 23117 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 10:21:22.343075 23122 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 10:21:22.343128 23117 server_base.cc:1034] running on GCE node
W20260619 10:21:22.343070 23123 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 10:21:22.343068 23125 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 10:21:22.343468 23117 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 10:21:22.343727 23117 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 10:21:22.344874 23117 hybrid_clock.cc:648] HybridClock initialized: now 1781864482344861 us; error 26 us; skew 500 ppm
Jun 19 10:21:22 dist-test-slave-6pt2 krb5kdc[22954](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864482, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 10:21:22.347216 23117 init.cc:377] Logged in from keytab as kudu/127.0.206.124@KRBTEST.COM (short username kudu)
I20260619 10:21:22.347558 23117 webserver.cc:460] Webserver started at http://127.0.206.124:33621/ using document root <none> and password file <none>
I20260619 10:21:22.347715 23117 fs_manager.cc:359] Metadata directory not provided
I20260619 10:21:22.347754 23117 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 10:21:22.347851 23117 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 10:21:22.348798 23117 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781864224577058-825-0/minicluster-data/master-2/data/instance:
uuid: "cd257ada5d4a47c4a69e0d5c1d532ff0"
format_stamp: "Formatted at 2026-06-19 10:21:22 on dist-test-slave-6pt2"
server_key: "7936fc86b873b2971a6c6d01dfd643d3"
server_key_iv: "144a7b1af6cffad80ea0a0c6fdf84d8e"
server_key_version: "encryptionkey@0"
I20260619 10:21:22.349071 23117 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781864224577058-825-0/minicluster-data/master-2/wal/instance:
uuid: "cd257ada5d4a47c4a69e0d5c1d532ff0"
format_stamp: "Formatted at 2026-06-19 10:21:22 on dist-test-slave-6pt2"
server_key: "7936fc86b873b2971a6c6d01dfd643d3"
server_key_iv: "144a7b1af6cffad80ea0a0c6fdf84d8e"
server_key_version: "encryptionkey@0"
I20260619 10:21:22.350309 23117 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 10:21:22.350955 23132 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 10:21:22.351131 23117 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260619 10:21:22.351191 23117 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781864224577058-825-0/minicluster-data/master-2/data,/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781864224577058-825-0/minicluster-data/master-2/wal
uuid: "cd257ada5d4a47c4a69e0d5c1d532ff0"
format_stamp: "Formatted at 2026-06-19 10:21:22 on dist-test-slave-6pt2"
server_key: "7936fc86b873b2971a6c6d01dfd643d3"
server_key_iv: "144a7b1af6cffad80ea0a0c6fdf84d8e"
server_key_version: "encryptionkey@0"
I20260619 10:21:22.351270 23117 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781864224577058-825-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781864224577058-825-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781864224577058-825-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 10:21:22.360308 23110 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.206.124" port: 45531 } attempt: 2
I20260619 10:21:22.360680 23112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:21:22.360518 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.124:45531 (local address 127.0.0.1:55882)
0619 10:21:22.360609 (+ 91us) negotiation.cc:107] Waiting for socket to connect
0619 10:21:22.360645 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.124:45531: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
I20260619 10:21:22.361085 23112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:21:22.360982 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.124:45531 (local address 127.0.0.1:55886)
0619 10:21:22.361030 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0619 10:21:22.361053 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.124:45531: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260619 10:21:22.361186 23110 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.206.124:45531: Network error: Client connection negotiation failed: client connection to 127.0.206.124:45531: connect: Connection refused (error 111)
I20260619 10:21:22.361770 23117 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 10:21:22.363423 23117 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 10:21:22.363565 23117 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 10:21:22.367302 23117 rpc_server.cc:307] RPC server started. Bound to: 127.0.206.124:45531
I20260619 10:21:22.367342 23184 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.206.124:45531 every 8 connection(s)
I20260619 10:21:22.367645 23117 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781864224577058-825-0/minicluster-data/master-2/data/info.pb
I20260619 10:21:22.368553 23185 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 10:21:22.369522 825 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTU9OP6/build/release/bin/kudu as pid 23117
I20260619 10:21:22.369622 825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781864224577058-825-0/minicluster-data/master-2/wal/instance
I20260619 10:21:22.369809 825 external_mini_cluster.cc:1428] Setting key 531cd6ac925998bd3046472bf5fc69f9
I20260619 10:21:22.370100 23185 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.206.126" port: 35571 } has no permanent_uuid. Determining permanent_uuid...
Jun 19 10:21:22 dist-test-slave-6pt2 krb5kdc[22954](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864482, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.124@KRBTEST.COM for kudu/127.0.206.126@KRBTEST.COM
Jun 19 10:21:22 dist-test-slave-6pt2 krb5kdc[22954](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864481, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.206.124@KRBTEST.COM
I20260619 10:21:22.378790 23038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:21:22.370713 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47204 (local address 127.0.206.126:35571)
0619 10:21:22.370750 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0619 10:21:22.370751 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 10:21:22.370939 (+ 188us) server_negotiation.cc:407] Connection header received
0619 10:21:22.371287 (+ 348us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:21:22.371288 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:21:22.371299 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:21:22.371355 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:21:22.371899 (+ 544us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:21:22.372401 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:21:22.373383 (+ 982us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:21:22.373671 (+ 288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:21:22.373742 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:21:22.375785 (+ 2043us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 10:21:22.375786 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 10:21:22.375786 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 10:21:22.375793 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 10:21:22.377338 (+ 1545us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:21:22.377809 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:21:22.377809 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:21:22.377810 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:21:22.377858 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:21:22.378109 (+ 251us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:21:22.378110 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:21:22.378110 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:21:22.378204 (+ 94us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 10:21:22.378296 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0619 10:21:22.378556 (+ 260us) server_negotiation.cc:299] Negotiation successful
0619 10:21:22.378680 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260619 10:21:22.378921 23189 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:21:22.370678 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:35571 (local address 127.0.0.1:47204)
0619 10:21:22.370835 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0619 10:21:22.370865 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0619 10:21:22.371170 (+ 305us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 10:21:22.371381 (+ 211us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 10:21:22.371383 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 10:21:22.371672 (+ 289us) client_negotiation.cc:189] Negotiated authn=SASL
0619 10:21:22.371812 (+ 140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:21:22.371815 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:21:22.372484 (+ 669us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:21:22.372485 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:21:22.373312 (+ 827us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:21:22.373314 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:21:22.373763 (+ 449us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:21:22.373764 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:21:22.373897 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:21:22.374294 (+ 397us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 10:21:22.374304 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0619 10:21:22.375675 (+ 1371us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 10:21:22.377423 (+ 1748us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 10:21:22.377429 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 10:21:22.377432 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 10:21:22.377737 (+ 305us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 10:21:22.377922 (+ 185us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 10:21:22.377923 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 10:21:22.377924 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 10:21:22.378043 (+ 119us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 10:21:22.378303 (+ 260us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 10:21:22.378306 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 10:21:22.378403 (+ 97us) client_negotiation.cc:769] Sending connection context
0619 10:21:22.378519 (+ 116us) client_negotiation.cc:240] Negotiation successful
0619 10:21:22.378695 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":51,"threads_started":1}
I20260619 10:21:22.379392 23185 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.206.125" port: 43989 } has no permanent_uuid. Determining permanent_uuid...
I20260619 10:21:22.379522 23188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:21:22.370264 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55898 (local address 127.0.206.124:45531)
0619 10:21:22.370558 (+ 294us) server_negotiation.cc:206] Beginning negotiation
0619 10:21:22.370565 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 10:21:22.370592 (+ 27us) server_negotiation.cc:407] Connection header received
0619 10:21:22.371167 (+ 575us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:21:22.371176 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:21:22.371210 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:21:22.371293 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:21:22.371989 (+ 696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:21:22.372923 (+ 934us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:21:22.373804 (+ 881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:21:22.374219 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:21:22.374286 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:21:22.376528 (+ 2242us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 10:21:22.376534 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 10:21:22.376534 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 10:21:22.376539 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0619 10:21:22.378180 (+ 1641us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:21:22.378509 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:21:22.378512 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:21:22.378512 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:21:22.378566 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:21:22.378967 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:21:22.378968 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:21:22.378968 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:21:22.379123 (+ 155us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 10:21:22.379254 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0619 10:21:22.379331 (+ 77us) server_negotiation.cc:299] Negotiation successful
0619 10:21:22.379393 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":255,"thread_start_us":70,"threads_started":1}
Jun 19 10:21:22 dist-test-slave-6pt2 krb5kdc[22954](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864482, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.124@KRBTEST.COM for kudu/127.0.206.125@KRBTEST.COM
Jun 19 10:21:22 dist-test-slave-6pt2 krb5kdc[22954](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864481, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.206.126@KRBTEST.COM
I20260619 10:21:22.385840 23189 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:21:22.379622 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.125:43989 (local address 127.0.0.1:40074)
0619 10:21:22.379654 (+ 32us) negotiation.cc:107] Waiting for socket to connect
0619 10:21:22.379662 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0619 10:21:22.379690 (+ 28us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 10:21:22.379839 (+ 149us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 10:21:22.379839 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 10:21:22.379974 (+ 135us) client_negotiation.cc:189] Negotiated authn=SASL
0619 10:21:22.380014 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:21:22.380015 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:21:22.380687 (+ 672us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:21:22.380687 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:21:22.381168 (+ 481us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:21:22.381169 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:21:22.381553 (+ 384us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:21:22.381554 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:21:22.381613 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:21:22.381965 (+ 352us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 10:21:22.381969 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 10:21:22.383172 (+ 1203us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 10:21:22.384855 (+ 1683us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 10:21:22.384856 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 10:21:22.384856 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 10:21:22.385033 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 10:21:22.385250 (+ 217us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 10:21:22.385250 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 10:21:22.385251 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 10:21:22.385302 (+ 51us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 10:21:22.385542 (+ 240us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 10:21:22.385543 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 10:21:22.385580 (+ 37us) client_negotiation.cc:769] Sending connection context
0619 10:21:22.385646 (+ 66us) client_negotiation.cc:240] Negotiation successful
0619 10:21:22.385751 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":22}
I20260619 10:21:22.385840 23114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:21:22.379673 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40074 (local address 127.0.206.125:43989)
0619 10:21:22.379710 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0619 10:21:22.379711 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 10:21:22.379738 (+ 27us) server_negotiation.cc:407] Connection header received
0619 10:21:22.379759 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:21:22.379760 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:21:22.379770 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:21:22.379819 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:21:22.380082 (+ 263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:21:22.380604 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:21:22.381255 (+ 651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:21:22.381478 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:21:22.381556 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:21:22.383254 (+ 1698us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 10:21:22.383255 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 10:21:22.383255 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 10:21:22.383262 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 10:21:22.384766 (+ 1504us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:21:22.385118 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:21:22.385119 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:21:22.385120 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:21:22.385170 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:21:22.385370 (+ 200us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:21:22.385371 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:21:22.385371 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:21:22.385479 (+ 108us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 10:21:22.385550 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0619 10:21:22.385674 (+ 124us) server_negotiation.cc:299] Negotiation successful
0619 10:21:22.385757 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260619 10:21:22.386574 23185 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.206.124" port: 45531 } has no permanent_uuid. Determining permanent_uuid...
I20260619 10:21:22.386729 23038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:21:22.380150 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47212 (local address 127.0.206.126:35571)
0619 10:21:22.380179 (+ 29us) server_negotiation.cc:206] Beginning negotiation
0619 10:21:22.380180 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 10:21:22.380186 (+ 6us) server_negotiation.cc:407] Connection header received
0619 10:21:22.380202 (+ 16us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:21:22.380203 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:21:22.380210 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:21:22.380267 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:21:22.380660 (+ 393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:21:22.381151 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:21:22.381822 (+ 671us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:21:22.382118 (+ 296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:21:22.382189 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:21:22.384226 (+ 2037us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 10:21:22.384227 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 10:21:22.384227 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 10:21:22.384234 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 10:21:22.385599 (+ 1365us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:21:22.385984 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:21:22.385985 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:21:22.385986 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:21:22.386051 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:21:22.386221 (+ 170us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:21:22.386222 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:21:22.386222 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:21:22.386342 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 10:21:22.386508 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0619 10:21:22.386548 (+ 40us) server_negotiation.cc:299] Negotiation successful
0619 10:21:22.386638 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
Jun 19 10:21:22 dist-test-slave-6pt2 krb5kdc[22954](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864482, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.124@KRBTEST.COM for kudu/127.0.206.124@KRBTEST.COM
Jun 19 10:21:22 dist-test-slave-6pt2 krb5kdc[22954](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864481, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.206.125@KRBTEST.COM
I20260619 10:21:22.392629 23189 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:21:22.386800 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.124:45531 (local address 127.0.0.1:55904)
0619 10:21:22.386836 (+ 36us) negotiation.cc:107] Waiting for socket to connect
0619 10:21:22.386861 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0619 10:21:22.386900 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 10:21:22.387116 (+ 216us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 10:21:22.387117 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 10:21:22.387253 (+ 136us) client_negotiation.cc:189] Negotiated authn=SASL
0619 10:21:22.387277 (+ 24us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:21:22.387278 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:21:22.387847 (+ 569us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:21:22.387848 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:21:22.388342 (+ 494us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:21:22.388344 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:21:22.388653 (+ 309us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:21:22.388653 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:21:22.388701 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:21:22.388985 (+ 284us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 10:21:22.388989 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 10:21:22.390180 (+ 1191us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 10:21:22.391678 (+ 1498us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 10:21:22.391679 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 10:21:22.391679 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 10:21:22.391871 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 10:21:22.392080 (+ 209us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 10:21:22.392080 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 10:21:22.392081 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 10:21:22.392129 (+ 48us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 10:21:22.392382 (+ 253us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 10:21:22.392383 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 10:21:22.392424 (+ 41us) client_negotiation.cc:769] Sending connection context
0619 10:21:22.392492 (+ 68us) client_negotiation.cc:240] Negotiation successful
0619 10:21:22.392578 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":21}
I20260619 10:21:22.392729 23188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:21:22.386961 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55904 (local address 127.0.206.124:45531)
0619 10:21:22.387012 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0619 10:21:22.387014 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 10:21:22.387021 (+ 7us) server_negotiation.cc:407] Connection header received
0619 10:21:22.387044 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:21:22.387045 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:21:22.387051 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:21:22.387115 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:21:22.387353 (+ 238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:21:22.387777 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:21:22.388411 (+ 634us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:21:22.388586 (+ 175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:21:22.388630 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:21:22.390266 (+ 1636us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 10:21:22.390267 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 10:21:22.390267 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 10:21:22.390273 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 10:21:22.391589 (+ 1316us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:21:22.391944 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:21:22.391945 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:21:22.391945 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:21:22.392010 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:21:22.392190 (+ 180us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:21:22.392191 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:21:22.392191 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:21:22.392311 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 10:21:22.392384 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0619 10:21:22.392530 (+ 146us) server_negotiation.cc:299] Negotiation successful
0619 10:21:22.392633 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":36,"server-negotiator.queue_time_us":37}
I20260619 10:21:22.394047 23185 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cd257ada5d4a47c4a69e0d5c1d532ff0: Bootstrap starting.
I20260619 10:21:22.394111 23114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:21:22.387358 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40076 (local address 127.0.206.125:43989)
0619 10:21:22.387395 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0619 10:21:22.387397 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 10:21:22.387406 (+ 9us) server_negotiation.cc:407] Connection header received
0619 10:21:22.387429 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:21:22.387430 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:21:22.387438 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:21:22.387489 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:21:22.387912 (+ 423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:21:22.388297 (+ 385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:21:22.389070 (+ 773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:21:22.389233 (+ 163us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:21:22.389277 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:21:22.391582 (+ 2305us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 10:21:22.391583 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 10:21:22.391583 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 10:21:22.391590 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 10:21:22.393044 (+ 1454us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:21:22.393399 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:21:22.393400 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:21:22.393401 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:21:22.393447 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:21:22.393642 (+ 195us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:21:22.393643 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:21:22.393644 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:21:22.393744 (+ 100us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 10:21:22.393898 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0619 10:21:22.393932 (+ 34us) server_negotiation.cc:299] Negotiation successful
0619 10:21:22.394004 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260619 10:21:22.395051 23185 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cd257ada5d4a47c4a69e0d5c1d532ff0: Neither blocks nor log segments found. Creating new log.
I20260619 10:21:22.395356 23185 log.cc:824] T 00000000000000000000000000000000 P cd257ada5d4a47c4a69e0d5c1d532ff0: Log is configured to *not* fsync() on all Append() calls
I20260619 10:21:22.395506 825 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 10:21:22.396647 23185 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cd257ada5d4a47c4a69e0d5c1d532ff0: No bootstrap required, opened a new log
I20260619 10:21:22.396478 23009 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "9e41ae07a1254bf38eadf9d3dd92205f"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47212
I20260619 10:21:22.398069 23185 raft_consensus.cc:348] T 00000000000000000000000000000000 P cd257ada5d4a47c4a69e0d5c1d532ff0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e41ae07a1254bf38eadf9d3dd92205f" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 35571 } } peers { permanent_uuid: "eab431b395514e4492e49d1d11a90a54" member_type: VOTER last_known_addr { host: "127.0.206.125" port: 43989 } } peers { permanent_uuid: "cd257ada5d4a47c4a69e0d5c1d532ff0" member_type: VOTER last_known_addr { host: "127.0.206.124" port: 45531 } }
I20260619 10:21:22.398176 23185 raft_consensus.cc:729] T 00000000000000000000000000000000 P cd257ada5d4a47c4a69e0d5c1d532ff0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cd257ada5d4a47c4a69e0d5c1d532ff0, State: Initialized, Role: FOLLOWER
I20260619 10:21:22.398259 23185 consensus_queue.cc:260] T 00000000000000000000000000000000 P cd257ada5d4a47c4a69e0d5c1d532ff0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e41ae07a1254bf38eadf9d3dd92205f" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 35571 } } peers { permanent_uuid: "eab431b395514e4492e49d1d11a90a54" member_type: VOTER last_known_addr { host: "127.0.206.125" port: 43989 } } peers { permanent_uuid: "cd257ada5d4a47c4a69e0d5c1d532ff0" member_type: VOTER last_known_addr { host: "127.0.206.124" port: 45531 } }
I20260619 10:21:22.398555 23191 sys_catalog.cc:455] T 00000000000000000000000000000000 P cd257ada5d4a47c4a69e0d5c1d532ff0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e41ae07a1254bf38eadf9d3dd92205f" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 35571 } } peers { permanent_uuid: "eab431b395514e4492e49d1d11a90a54" member_type: VOTER last_known_addr { host: "127.0.206.125" port: 43989 } } peers { permanent_uuid: "cd257ada5d4a47c4a69e0d5c1d532ff0" member_type: VOTER last_known_addr { host: "127.0.206.124" port: 45531 } } }
I20260619 10:21:22.398567 23185 sys_catalog.cc:564] T 00000000000000000000000000000000 P cd257ada5d4a47c4a69e0d5c1d532ff0 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 10:21:22.398705 23191 sys_catalog.cc:458] T 00000000000000000000000000000000 P cd257ada5d4a47c4a69e0d5c1d532ff0 [sys.catalog]: This master's current role is: FOLLOWER
W20260619 10:21:22.400430 23202 catalog_manager.cc:1558] T 00000000000000000000000000000000 P cd257ada5d4a47c4a69e0d5c1d532ff0: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 10:21:22.400484 23202 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 10:21:22.400645 23117 master_runner.cc:186] Error getting master registration for 127.0.206.126:35571: OK, instance_id { permanent_uuid: "9e41ae07a1254bf38eadf9d3dd92205f" instance_seqno: 1781864482065245 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260619 10:21:22.403308 23117 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260619 10:21:22.406929 23035 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.206.124" port: 45531 } attempt: 2
Jun 19 10:21:22 dist-test-slave-6pt2 krb5kdc[22954](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864482, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.126@KRBTEST.COM for kudu/127.0.206.124@KRBTEST.COM
I20260619 10:21:22.413612 23188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:21:22.407350 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55908 (local address 127.0.206.124:45531)
0619 10:21:22.407390 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0619 10:21:22.407392 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 10:21:22.407398 (+ 6us) server_negotiation.cc:407] Connection header received
0619 10:21:22.407415 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:21:22.407415 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:21:22.407423 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:21:22.407494 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:21:22.407739 (+ 245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:21:22.408203 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:21:22.408874 (+ 671us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:21:22.409079 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:21:22.409152 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:21:22.410940 (+ 1788us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 10:21:22.410941 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 10:21:22.410942 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 10:21:22.410949 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 10:21:22.412463 (+ 1514us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:21:22.412841 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:21:22.412842 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:21:22.412842 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:21:22.412893 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:21:22.413082 (+ 189us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:21:22.413083 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:21:22.413083 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:21:22.413221 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 10:21:22.413289 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0619 10:21:22.413457 (+ 168us) server_negotiation.cc:299] Negotiation successful
0619 10:21:22.413526 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260619 10:21:22.413614 23037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:21:22.407185 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.124:45531 (local address 127.0.0.1:55908)
0619 10:21:22.407235 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0619 10:21:22.407248 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 10:21:22.407283 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 10:21:22.407498 (+ 215us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 10:21:22.407499 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 10:21:22.407637 (+ 138us) client_negotiation.cc:189] Negotiated authn=SASL
0619 10:21:22.407667 (+ 30us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:21:22.407669 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:21:22.408296 (+ 627us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:21:22.408297 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:21:22.408796 (+ 499us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:21:22.408797 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:21:22.409151 (+ 354us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:21:22.409152 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:21:22.409207 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:21:22.409605 (+ 398us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 10:21:22.409610 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 10:21:22.410864 (+ 1254us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 10:21:22.412560 (+ 1696us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 10:21:22.412561 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 10:21:22.412561 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 10:21:22.412769 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 10:21:22.412962 (+ 193us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 10:21:22.412963 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 10:21:22.412963 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 10:21:22.413023 (+ 60us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 10:21:22.413294 (+ 271us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 10:21:22.413295 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 10:21:22.413337 (+ 42us) client_negotiation.cc:769] Sending connection context
0619 10:21:22.413405 (+ 68us) client_negotiation.cc:240] Negotiation successful
0619 10:21:22.413510 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":20}
I20260619 10:21:22.415063 23035 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9e41ae07a1254bf38eadf9d3dd92205f: Bootstrap starting.
I20260619 10:21:22.415724 23035 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9e41ae07a1254bf38eadf9d3dd92205f: Neither blocks nor log segments found. Creating new log.
I20260619 10:21:22.416021 23035 log.cc:824] T 00000000000000000000000000000000 P 9e41ae07a1254bf38eadf9d3dd92205f: Log is configured to *not* fsync() on all Append() calls
I20260619 10:21:22.416775 23035 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9e41ae07a1254bf38eadf9d3dd92205f: No bootstrap required, opened a new log
I20260619 10:21:22.418108 23035 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9e41ae07a1254bf38eadf9d3dd92205f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e41ae07a1254bf38eadf9d3dd92205f" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 35571 } } peers { permanent_uuid: "eab431b395514e4492e49d1d11a90a54" member_type: VOTER last_known_addr { host: "127.0.206.125" port: 43989 } } peers { permanent_uuid: "cd257ada5d4a47c4a69e0d5c1d532ff0" member_type: VOTER last_known_addr { host: "127.0.206.124" port: 45531 } }
I20260619 10:21:22.418236 23035 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9e41ae07a1254bf38eadf9d3dd92205f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e41ae07a1254bf38eadf9d3dd92205f, State: Initialized, Role: FOLLOWER
I20260619 10:21:22.418316 23035 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9e41ae07a1254bf38eadf9d3dd92205f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e41ae07a1254bf38eadf9d3dd92205f" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 35571 } } peers { permanent_uuid: "eab431b395514e4492e49d1d11a90a54" member_type: VOTER last_known_addr { host: "127.0.206.125" port: 43989 } } peers { permanent_uuid: "cd257ada5d4a47c4a69e0d5c1d532ff0" member_type: VOTER last_known_addr { host: "127.0.206.124" port: 45531 } }
I20260619 10:21:22.418637 23035 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9e41ae07a1254bf38eadf9d3dd92205f [sys.catalog]: configured and running, proceeding with master startup.
I20260619 10:21:22.418591 23205 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9e41ae07a1254bf38eadf9d3dd92205f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e41ae07a1254bf38eadf9d3dd92205f" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 35571 } } peers { permanent_uuid: "eab431b395514e4492e49d1d11a90a54" member_type: VOTER last_known_addr { host: "127.0.206.125" port: 43989 } } peers { permanent_uuid: "cd257ada5d4a47c4a69e0d5c1d532ff0" member_type: VOTER last_known_addr { host: "127.0.206.124" port: 45531 } } }
I20260619 10:21:22.418702 23205 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9e41ae07a1254bf38eadf9d3dd92205f [sys.catalog]: This master's current role is: FOLLOWER
I20260619 10:21:22.418740 23009 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9e41ae07a1254bf38eadf9d3dd92205f [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260619 10:21:22.418898 23009 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9e41ae07a1254bf38eadf9d3dd92205f [term 0 FOLLOWER]: Advancing to term 1
I20260619 10:21:22.419730 23009 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9e41ae07a1254bf38eadf9d3dd92205f [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e41ae07a1254bf38eadf9d3dd92205f" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 35571 } } peers { permanent_uuid: "eab431b395514e4492e49d1d11a90a54" member_type: VOTER last_known_addr { host: "127.0.206.125" port: 43989 } } peers { permanent_uuid: "cd257ada5d4a47c4a69e0d5c1d532ff0" member_type: VOTER last_known_addr { host: "127.0.206.124" port: 45531 } }
W20260619 10:21:22.420498 23216 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 9e41ae07a1254bf38eadf9d3dd92205f: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 10:21:22.420559 23216 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 10:21:22.421450 22967 master_runner.cc:186] Error getting master registration for 127.0.206.125:43989: OK, instance_id { permanent_uuid: "eab431b395514e4492e49d1d11a90a54" instance_seqno: 1781864482219571 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260619 10:21:22.423588 22967 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260619 10:21:22.423839 23009 leader_election.cc:288] T 00000000000000000000000000000000 P 9e41ae07a1254bf38eadf9d3dd92205f [CANDIDATE]: Term 1 election: Requested vote from peers eab431b395514e4492e49d1d11a90a54 (127.0.206.125:43989), cd257ada5d4a47c4a69e0d5c1d532ff0 (127.0.206.124:45531)
I20260619 10:21:22.423933 23159 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "9e41ae07a1254bf38eadf9d3dd92205f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "cd257ada5d4a47c4a69e0d5c1d532ff0"
I20260619 10:21:22.424058 23159 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cd257ada5d4a47c4a69e0d5c1d532ff0 [term 0 FOLLOWER]: Advancing to term 1
I20260619 10:21:22.424898 23159 raft_consensus.cc:2454] T 00000000000000000000000000000000 P cd257ada5d4a47c4a69e0d5c1d532ff0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e41ae07a1254bf38eadf9d3dd92205f in term 1.
I20260619 10:21:22.425107 22985 leader_election.cc:302] T 00000000000000000000000000000000 P 9e41ae07a1254bf38eadf9d3dd92205f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e41ae07a1254bf38eadf9d3dd92205f, cd257ada5d4a47c4a69e0d5c1d532ff0; no voters:
I20260619 10:21:22.425223 23205 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9e41ae07a1254bf38eadf9d3dd92205f [term 1 FOLLOWER]: Leader election won for term 1
I20260619 10:21:22.425355 23205 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9e41ae07a1254bf38eadf9d3dd92205f [term 1 LEADER]: Becoming Leader. State: Replica: 9e41ae07a1254bf38eadf9d3dd92205f, State: Running, Role: LEADER
I20260619 10:21:22.425428 23205 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9e41ae07a1254bf38eadf9d3dd92205f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e41ae07a1254bf38eadf9d3dd92205f" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 35571 } } peers { permanent_uuid: "eab431b395514e4492e49d1d11a90a54" member_type: VOTER last_known_addr { host: "127.0.206.125" port: 43989 } } peers { permanent_uuid: "cd257ada5d4a47c4a69e0d5c1d532ff0" member_type: VOTER last_known_addr { host: "127.0.206.124" port: 45531 } }
I20260619 10:21:22.425809 23205 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9e41ae07a1254bf38eadf9d3dd92205f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9e41ae07a1254bf38eadf9d3dd92205f. Latest consensus state: current_term: 1 leader_uuid: "9e41ae07a1254bf38eadf9d3dd92205f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e41ae07a1254bf38eadf9d3dd92205f" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 35571 } } peers { permanent_uuid: "eab431b395514e4492e49d1d11a90a54" member_type: VOTER last_known_addr { host: "127.0.206.125" port: 43989 } } peers { permanent_uuid: "cd257ada5d4a47c4a69e0d5c1d532ff0" member_type: VOTER last_known_addr { host: "127.0.206.124" port: 45531 } } }
I20260619 10:21:22.425895 23205 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9e41ae07a1254bf38eadf9d3dd92205f [sys.catalog]: This master's current role is: LEADER
I20260619 10:21:22.430533 23224 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 10:21:22.430776 23224 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 10:21:22.433912 23188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:21:22.425819 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55914 (local address 127.0.206.124:45531)
0619 10:21:22.425859 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0619 10:21:22.425861 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 10:21:22.426310 (+ 449us) server_negotiation.cc:407] Connection header received
0619 10:21:22.426411 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:21:22.426412 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:21:22.426422 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:21:22.426451 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:21:22.427248 (+ 797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:21:22.427974 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:21:22.429019 (+ 1045us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:21:22.429287 (+ 268us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:21:22.429326 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:21:22.431022 (+ 1696us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 10:21:22.431023 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 10:21:22.431023 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 10:21:22.431030 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 10:21:22.432694 (+ 1664us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:21:22.433104 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:21:22.433105 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:21:22.433105 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:21:22.433157 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:21:22.433363 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:21:22.433364 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:21:22.433364 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:21:22.433487 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 10:21:22.433553 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0619 10:21:22.433753 (+ 200us) server_negotiation.cc:299] Negotiation successful
0619 10:21:22.433820 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260619 10:21:22.439846 23038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:21:22.425406 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47216 (local address 127.0.206.126:35571)
0619 10:21:22.425453 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0619 10:21:22.425455 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 10:21:22.427237 (+ 1782us) server_negotiation.cc:407] Connection header received
0619 10:21:22.427440 (+ 203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:21:22.427441 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:21:22.427450 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:21:22.427490 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:21:22.430260 (+ 2770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:21:22.431000 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:21:22.433769 (+ 2769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:21:22.434062 (+ 293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:21:22.434101 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:21:22.436513 (+ 2412us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 10:21:22.436514 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 10:21:22.436515 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 10:21:22.436522 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 10:21:22.438588 (+ 2066us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:21:22.439052 (+ 464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:21:22.439053 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:21:22.439054 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:21:22.439106 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:21:22.439286 (+ 180us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:21:22.439286 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:21:22.439287 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:21:22.439398 (+ 111us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 10:21:22.439488 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0619 10:21:22.439687 (+ 199us) server_negotiation.cc:299] Negotiation successful
0619 10:21:22.439756 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260619 10:21:22.440336 23114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:21:22.425645 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40080 (local address 127.0.206.125:43989)
0619 10:21:22.425686 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0619 10:21:22.425688 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 10:21:22.427172 (+ 1484us) server_negotiation.cc:407] Connection header received
0619 10:21:22.427579 (+ 407us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:21:22.427580 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:21:22.427590 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:21:22.427618 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:21:22.429361 (+ 1743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:21:22.430114 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:21:22.434224 (+ 4110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:21:22.434570 (+ 346us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:21:22.434624 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:21:22.437010 (+ 2386us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 10:21:22.437011 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 10:21:22.437012 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 10:21:22.437029 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0619 10:21:22.438697 (+ 1668us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:21:22.439297 (+ 600us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:21:22.439298 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:21:22.439299 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:21:22.439364 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:21:22.439629 (+ 265us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:21:22.439629 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:21:22.439630 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:21:22.439764 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 10:21:22.439830 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0619 10:21:22.440165 (+ 335us) server_negotiation.cc:299] Negotiation successful
0619 10:21:22.440243 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260619 10:21:22.447479 23159 raft_consensus.cc:1261] T 00000000000000000000000000000000 P cd257ada5d4a47c4a69e0d5c1d532ff0 [term 1 FOLLOWER]: Refusing update from remote peer 9e41ae07a1254bf38eadf9d3dd92205f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 10:21:22.447803 23222 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 9e41ae07a1254bf38eadf9d3dd92205f [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd257ada5d4a47c4a69e0d5c1d532ff0" member_type: VOTER last_known_addr { host: "127.0.206.124" port: 45531 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 10:21:22.449095 23191 sys_catalog.cc:455] T 00000000000000000000000000000000 P cd257ada5d4a47c4a69e0d5c1d532ff0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9e41ae07a1254bf38eadf9d3dd92205f. Latest consensus state: current_term: 1 leader_uuid: "9e41ae07a1254bf38eadf9d3dd92205f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e41ae07a1254bf38eadf9d3dd92205f" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 35571 } } peers { permanent_uuid: "eab431b395514e4492e49d1d11a90a54" member_type: VOTER last_known_addr { host: "127.0.206.125" port: 43989 } } peers { permanent_uuid: "cd257ada5d4a47c4a69e0d5c1d532ff0" member_type: VOTER last_known_addr { host: "127.0.206.124" port: 45531 } } }
I20260619 10:21:22.449200 23191 sys_catalog.cc:458] T 00000000000000000000000000000000 P cd257ada5d4a47c4a69e0d5c1d532ff0 [sys.catalog]: This master's current role is: FOLLOWER
I20260619 10:21:22.451030 23205 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9e41ae07a1254bf38eadf9d3dd92205f [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "9e41ae07a1254bf38eadf9d3dd92205f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e41ae07a1254bf38eadf9d3dd92205f" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 35571 } } peers { permanent_uuid: "eab431b395514e4492e49d1d11a90a54" member_type: VOTER last_known_addr { host: "127.0.206.125" port: 43989 } } peers { permanent_uuid: "cd257ada5d4a47c4a69e0d5c1d532ff0" member_type: VOTER last_known_addr { host: "127.0.206.124" port: 45531 } } }
I20260619 10:21:22.451105 23205 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9e41ae07a1254bf38eadf9d3dd92205f [sys.catalog]: This master's current role is: LEADER
I20260619 10:21:22.451186 23191 sys_catalog.cc:455] T 00000000000000000000000000000000 P cd257ada5d4a47c4a69e0d5c1d532ff0 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "9e41ae07a1254bf38eadf9d3dd92205f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e41ae07a1254bf38eadf9d3dd92205f" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 35571 } } peers { permanent_uuid: "eab431b395514e4492e49d1d11a90a54" member_type: VOTER last_known_addr { host: "127.0.206.125" port: 43989 } } peers { permanent_uuid: "cd257ada5d4a47c4a69e0d5c1d532ff0" member_type: VOTER last_known_addr { host: "127.0.206.124" port: 45531 } } }
I20260619 10:21:22.451256 23191 sys_catalog.cc:458] T 00000000000000000000000000000000 P cd257ada5d4a47c4a69e0d5c1d532ff0 [sys.catalog]: This master's current role is: FOLLOWER
I20260619 10:21:22.451474 23229 mvcc.cc:205] Tried to move back new op lower bound from 7298516920102252544 to 7298516920015245312. Current Snapshot: MvccSnapshot[applied={T|T < 7298516920102252544}]
I20260619 10:21:22.452312 23110 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.206.124" port: 45531 } attempt: 3
I20260619 10:21:22.455241 23224 catalog_manager.cc:1347] Generated new cluster ID: 87fb1ca0c32446d8b047d2943ec53298
I20260619 10:21:22.455286 23224 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Jun 19 10:21:22 dist-test-slave-6pt2 krb5kdc[22954](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864482, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.125@KRBTEST.COM for kudu/127.0.206.124@KRBTEST.COM
I20260619 10:21:22.461014 23112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:21:22.452558 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.124:45531 (local address 127.0.0.1:55928)
0619 10:21:22.452598 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0619 10:21:22.452607 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0619 10:21:22.452635 (+ 28us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 10:21:22.452787 (+ 152us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 10:21:22.452788 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 10:21:22.452943 (+ 155us) client_negotiation.cc:189] Negotiated authn=SASL
0619 10:21:22.452986 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:21:22.452988 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:21:22.453923 (+ 935us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:21:22.453924 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:21:22.454774 (+ 850us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:21:22.454777 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:21:22.455290 (+ 513us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:21:22.455291 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:21:22.455356 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:21:22.455939 (+ 583us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 10:21:22.455956 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0619 10:21:22.457995 (+ 2039us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 10:21:22.459925 (+ 1930us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 10:21:22.459925 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 10:21:22.459926 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 10:21:22.460132 (+ 206us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 10:21:22.460355 (+ 223us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 10:21:22.460356 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 10:21:22.460356 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 10:21:22.460419 (+ 63us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 10:21:22.460712 (+ 293us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 10:21:22.460712 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 10:21:22.460766 (+ 54us) client_negotiation.cc:769] Sending connection context
0619 10:21:22.460823 (+ 57us) client_negotiation.cc:240] Negotiation successful
0619 10:21:22.460908 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":25}
I20260619 10:21:22.461431 23188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:21:22.452579 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55928 (local address 127.0.206.124:45531)
0619 10:21:22.452615 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0619 10:21:22.452617 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 10:21:22.452642 (+ 25us) server_negotiation.cc:407] Connection header received
0619 10:21:22.452698 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:21:22.452699 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:21:22.452709 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:21:22.452751 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:21:22.453055 (+ 304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:21:22.453799 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:21:22.454865 (+ 1066us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:21:22.455161 (+ 296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:21:22.455215 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:21:22.458093 (+ 2878us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 10:21:22.458095 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 10:21:22.458095 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 10:21:22.458101 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 10:21:22.459826 (+ 1725us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:21:22.460219 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:21:22.460220 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:21:22.460220 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:21:22.460283 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:21:22.460495 (+ 212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:21:22.460496 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:21:22.460497 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:21:22.460624 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 10:21:22.460677 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0619 10:21:22.461278 (+ 601us) server_negotiation.cc:299] Negotiation successful
0619 10:21:22.461356 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260619 10:21:22.463167 23110 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eab431b395514e4492e49d1d11a90a54: Bootstrap starting.
I20260619 10:21:22.463833 23110 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P eab431b395514e4492e49d1d11a90a54: Neither blocks nor log segments found. Creating new log.
I20260619 10:21:22.464112 23110 log.cc:824] T 00000000000000000000000000000000 P eab431b395514e4492e49d1d11a90a54: Log is configured to *not* fsync() on all Append() calls
I20260619 10:21:22.471248 23110 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eab431b395514e4492e49d1d11a90a54: No bootstrap required, opened a new log
I20260619 10:21:22.473259 23110 raft_consensus.cc:348] T 00000000000000000000000000000000 P eab431b395514e4492e49d1d11a90a54 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e41ae07a1254bf38eadf9d3dd92205f" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 35571 } } peers { permanent_uuid: "eab431b395514e4492e49d1d11a90a54" member_type: VOTER last_known_addr { host: "127.0.206.125" port: 43989 } } peers { permanent_uuid: "cd257ada5d4a47c4a69e0d5c1d532ff0" member_type: VOTER last_known_addr { host: "127.0.206.124" port: 45531 } }
I20260619 10:21:22.473381 23110 raft_consensus.cc:729] T 00000000000000000000000000000000 P eab431b395514e4492e49d1d11a90a54 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eab431b395514e4492e49d1d11a90a54, State: Initialized, Role: FOLLOWER
I20260619 10:21:22.473450 23110 consensus_queue.cc:260] T 00000000000000000000000000000000 P eab431b395514e4492e49d1d11a90a54 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e41ae07a1254bf38eadf9d3dd92205f" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 35571 } } peers { permanent_uuid: "eab431b395514e4492e49d1d11a90a54" member_type: VOTER last_known_addr { host: "127.0.206.125" port: 43989 } } peers { permanent_uuid: "cd257ada5d4a47c4a69e0d5c1d532ff0" member_type: VOTER last_known_addr { host: "127.0.206.124" port: 45531 } }
I20260619 10:21:22.473953 23224 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 10:21:22.474416 23083 raft_consensus.cc:3037] T 00000000000000000000000000000000 P eab431b395514e4492e49d1d11a90a54 [term 0 FOLLOWER]: Advancing to term 1
I20260619 10:21:22.474689 23224 catalog_manager.cc:1504] Loading token signing keys...
I20260619 10:21:22.475296 23083 raft_consensus.cc:1261] T 00000000000000000000000000000000 P eab431b395514e4492e49d1d11a90a54 [term 1 FOLLOWER]: Refusing update from remote peer 9e41ae07a1254bf38eadf9d3dd92205f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 10:21:22.475569 23205 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 9e41ae07a1254bf38eadf9d3dd92205f [LEADER]: Connected to new peer: Peer: permanent_uuid: "eab431b395514e4492e49d1d11a90a54" member_type: VOTER last_known_addr { host: "127.0.206.125" port: 43989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 10:21:22.474303 23084 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "9e41ae07a1254bf38eadf9d3dd92205f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "eab431b395514e4492e49d1d11a90a54"
I20260619 10:21:22.476613 23084 raft_consensus.cc:2454] T 00000000000000000000000000000000 P eab431b395514e4492e49d1d11a90a54 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e41ae07a1254bf38eadf9d3dd92205f in term 1.
I20260619 10:21:22.478533 23110 sys_catalog.cc:564] T 00000000000000000000000000000000 P eab431b395514e4492e49d1d11a90a54 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 10:21:22.482690 23235 sys_catalog.cc:455] T 00000000000000000000000000000000 P eab431b395514e4492e49d1d11a90a54 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9e41ae07a1254bf38eadf9d3dd92205f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e41ae07a1254bf38eadf9d3dd92205f" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 35571 } } peers { permanent_uuid: "eab431b395514e4492e49d1d11a90a54" member_type: VOTER last_known_addr { host: "127.0.206.125" port: 43989 } } peers { permanent_uuid: "cd257ada5d4a47c4a69e0d5c1d532ff0" member_type: VOTER last_known_addr { host: "127.0.206.124" port: 45531 } } }
I20260619 10:21:22.482776 23235 sys_catalog.cc:458] T 00000000000000000000000000000000 P eab431b395514e4492e49d1d11a90a54 [sys.catalog]: This master's current role is: FOLLOWER
I20260619 10:21:22.482825 23235 sys_catalog.cc:455] T 00000000000000000000000000000000 P eab431b395514e4492e49d1d11a90a54 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9e41ae07a1254bf38eadf9d3dd92205f. Latest consensus state: current_term: 1 leader_uuid: "9e41ae07a1254bf38eadf9d3dd92205f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e41ae07a1254bf38eadf9d3dd92205f" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 35571 } } peers { permanent_uuid: "eab431b395514e4492e49d1d11a90a54" member_type: VOTER last_known_addr { host: "127.0.206.125" port: 43989 } } peers { permanent_uuid: "cd257ada5d4a47c4a69e0d5c1d532ff0" member_type: VOTER last_known_addr { host: "127.0.206.124" port: 45531 } } }
I20260619 10:21:22.482873 23235 sys_catalog.cc:458] T 00000000000000000000000000000000 P eab431b395514e4492e49d1d11a90a54 [sys.catalog]: This master's current role is: FOLLOWER
I20260619 10:21:22.482908 23235 sys_catalog.cc:455] T 00000000000000000000000000000000 P eab431b395514e4492e49d1d11a90a54 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "9e41ae07a1254bf38eadf9d3dd92205f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e41ae07a1254bf38eadf9d3dd92205f" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 35571 } } peers { permanent_uuid: "eab431b395514e4492e49d1d11a90a54" member_type: VOTER last_known_addr { host: "127.0.206.125" port: 43989 } } peers { permanent_uuid: "cd257ada5d4a47c4a69e0d5c1d532ff0" member_type: VOTER last_known_addr { host: "127.0.206.124" port: 45531 } } }
I20260619 10:21:22.482947 23235 sys_catalog.cc:458] T 00000000000000000000000000000000 P eab431b395514e4492e49d1d11a90a54 [sys.catalog]: This master's current role is: FOLLOWER
I20260619 10:21:22.483325 23236 mvcc.cc:205] Tried to move back new op lower bound from 7298516920194605056 to 7298516920015245312. Current Snapshot: MvccSnapshot[applied={T|T < 7298516920102252544}]
I20260619 10:21:22.485862 23205 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9e41ae07a1254bf38eadf9d3dd92205f [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "9e41ae07a1254bf38eadf9d3dd92205f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e41ae07a1254bf38eadf9d3dd92205f" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 35571 } } peers { permanent_uuid: "eab431b395514e4492e49d1d11a90a54" member_type: VOTER last_known_addr { host: "127.0.206.125" port: 43989 } } peers { permanent_uuid: "cd257ada5d4a47c4a69e0d5c1d532ff0" member_type: VOTER last_known_addr { host: "127.0.206.124" port: 45531 } } }
I20260619 10:21:22.485960 23205 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9e41ae07a1254bf38eadf9d3dd92205f [sys.catalog]: This master's current role is: LEADER
I20260619 10:21:22.487782 23224 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9e41ae07a1254bf38eadf9d3dd92205f: Generated new TSK 0
I20260619 10:21:22.487926 23224 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 10:21:22.491334 23249 catalog_manager.cc:1259] Loaded cluster ID: 87fb1ca0c32446d8b047d2943ec53298
I20260619 10:21:22.491380 23249 catalog_manager.cc:1552] T 00000000000000000000000000000000 P eab431b395514e4492e49d1d11a90a54: loading cluster ID for follower catalog manager: success
I20260619 10:21:22.492430 23249 catalog_manager.cc:1574] T 00000000000000000000000000000000 P eab431b395514e4492e49d1d11a90a54: acquiring CA information for follower catalog manager: success
I20260619 10:21:22.492722 23249 catalog_manager.cc:1602] T 00000000000000000000000000000000 P eab431b395514e4492e49d1d11a90a54: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260619 10:21:22.519155 23114 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 10:21:22.517640 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40094 (local address 127.0.206.125:43989)
0619 10:21:22.517683 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0619 10:21:22.517685 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 10:21:22.518518 (+ 833us) server_negotiation.cc:407] Connection header received
0619 10:21:22.518651 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:21:22.518652 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:21:22.518663 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:21:22.518692 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:21:22.519084 (+ 392us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40094: BlockingRecv error: recv got EOF from 127.0.0.1:40094 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
W20260619 10:21:22.519843 23188 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 10:21:22.517840 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55942 (local address 127.0.206.124:45531)
0619 10:21:22.517876 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0619 10:21:22.517878 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 10:21:22.519402 (+ 1524us) server_negotiation.cc:407] Connection header received
0619 10:21:22.519502 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:21:22.519503 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:21:22.519512 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:21:22.519537 (+ 25us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:21:22.519794 (+ 257us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55942: BlockingRecv error: recv got EOF from 127.0.0.1:55942 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
W20260619 10:21:22.520498 23038 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 10:21:22.518031 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47226 (local address 127.0.206.126:35571)
0619 10:21:22.518067 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0619 10:21:22.518069 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 10:21:22.520055 (+ 1986us) server_negotiation.cc:407] Connection header received
0619 10:21:22.520139 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:21:22.520140 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:21:22.520149 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:21:22.520175 (+ 26us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:21:22.520444 (+ 269us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47226: BlockingRecv error: recv got EOF from 127.0.0.1:47226 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
I20260619 10:21:22.525317 23114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:21:22.522051 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40096 (local address 127.0.206.125:43989)
0619 10:21:22.522083 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0619 10:21:22.522085 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 10:21:22.522300 (+ 215us) server_negotiation.cc:407] Connection header received
0619 10:21:22.522395 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:21:22.522396 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:21:22.522427 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:21:22.522467 (+ 40us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 10:21:22.522702 (+ 235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:21:22.523431 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:21:22.524458 (+ 1027us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:21:22.524722 (+ 264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:21:22.524777 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:21:22.524992 (+ 215us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 10:21:22.525094 (+ 102us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 10:21:22.525141 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0619 10:21:22.525249 (+ 108us) server_negotiation.cc:299] Negotiation successful
0619 10:21:22.525264 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
W20260619 10:21:22.528286 23188 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 10:21:22.527567 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55954 (local address 127.0.206.124:45531)
0619 10:21:22.527599 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0619 10:21:22.527600 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 10:21:22.527782 (+ 182us) server_negotiation.cc:407] Connection header received
0619 10:21:22.527879 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:21:22.527880 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:21:22.527894 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:21:22.527922 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:21:22.528222 (+ 300us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55954: BlockingRecv error: recv got EOF from 127.0.0.1:55954 (error 108)
Metrics: {"server-negotiator.queue_time_us":16}
I20260619 10:21:22.533294 23114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:21:22.530391 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40100 (local address 127.0.206.125:43989)
0619 10:21:22.530421 (+ 30us) server_negotiation.cc:206] Beginning negotiation
0619 10:21:22.530423 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 10:21:22.530645 (+ 222us) server_negotiation.cc:407] Connection header received
0619 10:21:22.530741 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:21:22.530742 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:21:22.530750 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:21:22.530776 (+ 26us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 10:21:22.530936 (+ 160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:21:22.531579 (+ 643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:21:22.532598 (+ 1019us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:21:22.532846 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:21:22.532887 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:21:22.533078 (+ 191us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 10:21:22.533105 (+ 27us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 10:21:22.533130 (+ 25us) server_negotiation.cc:1035] Waiting for connection context
0619 10:21:22.533225 (+ 95us) server_negotiation.cc:299] Negotiation successful
0619 10:21:22.533241 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
W20260619 10:21:22.536046 23188 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 10:21:22.535326 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55960 (local address 127.0.206.124:45531)
0619 10:21:22.535357 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0619 10:21:22.535358 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 10:21:22.535550 (+ 192us) server_negotiation.cc:407] Connection header received
0619 10:21:22.535640 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:21:22.535641 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:21:22.535650 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:21:22.535681 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:21:22.536001 (+ 320us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55960: BlockingRecv error: recv got EOF from 127.0.0.1:55960 (error 108)
Metrics: {"server-negotiator.queue_time_us":16}
I20260619 10:21:22.542217 23114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:21:22.539171 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40108 (local address 127.0.206.125:43989)
0619 10:21:22.539203 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0619 10:21:22.539205 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 10:21:22.539416 (+ 211us) server_negotiation.cc:407] Connection header received
0619 10:21:22.539509 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:21:22.539510 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:21:22.539518 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:21:22.539544 (+ 26us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 10:21:22.539701 (+ 157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:21:22.540375 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:21:22.541438 (+ 1063us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:21:22.541704 (+ 266us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:21:22.541749 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:21:22.541942 (+ 193us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 10:21:22.542004 (+ 62us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 10:21:22.542033 (+ 29us) server_negotiation.cc:1035] Waiting for connection context
0619 10:21:22.542140 (+ 107us) server_negotiation.cc:299] Negotiation successful
0619 10:21:22.542156 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
W20260619 10:21:22.545109 23188 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 10:21:22.544292 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55974 (local address 127.0.206.124:45531)
0619 10:21:22.544324 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0619 10:21:22.544326 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 10:21:22.544579 (+ 253us) server_negotiation.cc:407] Connection header received
0619 10:21:22.544705 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:21:22.544706 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:21:22.544716 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:21:22.544744 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:21:22.545062 (+ 318us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55974: BlockingRecv error: recv got EOF from 127.0.0.1:55974 (error 108)
Metrics: {"server-negotiator.queue_time_us":16}
I20260619 10:21:22.553134 23114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:21:22.550136 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40122 (local address 127.0.206.125:43989)
0619 10:21:22.550168 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0619 10:21:22.550169 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 10:21:22.550374 (+ 205us) server_negotiation.cc:407] Connection header received
0619 10:21:22.550462 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:21:22.550463 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:21:22.550472 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:21:22.550499 (+ 27us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 10:21:22.550678 (+ 179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:21:22.551337 (+ 659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:21:22.552346 (+ 1009us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:21:22.552605 (+ 259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:21:22.552646 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:21:22.552834 (+ 188us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 10:21:22.552863 (+ 29us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 10:21:22.552890 (+ 27us) server_negotiation.cc:1035] Waiting for connection context
0619 10:21:22.553057 (+ 167us) server_negotiation.cc:299] Negotiation successful
0619 10:21:22.553074 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
W20260619 10:21:22.556025 23188 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 10:21:22.555312 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55976 (local address 127.0.206.124:45531)
0619 10:21:22.555344 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0619 10:21:22.555345 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 10:21:22.555526 (+ 181us) server_negotiation.cc:407] Connection header received
0619 10:21:22.555617 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:21:22.555618 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:21:22.555627 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:21:22.555655 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:21:22.555980 (+ 325us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55976: BlockingRecv error: recv got EOF from 127.0.0.1:55976 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
I20260619 10:21:22.568264 23114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:21:22.565057 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40126 (local address 127.0.206.125:43989)
0619 10:21:22.565092 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0619 10:21:22.565094 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 10:21:22.565320 (+ 226us) server_negotiation.cc:407] Connection header received
0619 10:21:22.565407 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:21:22.565408 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:21:22.565417 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:21:22.565443 (+ 26us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 10:21:22.565600 (+ 157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:21:22.566301 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:21:22.567482 (+ 1181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:21:22.567763 (+ 281us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:21:22.567802 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:21:22.568006 (+ 204us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 10:21:22.568036 (+ 30us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 10:21:22.568064 (+ 28us) server_negotiation.cc:1035] Waiting for connection context
0619 10:21:22.568169 (+ 105us) server_negotiation.cc:299] Negotiation successful
0619 10:21:22.568185 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
W20260619 10:21:22.571092 23188 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 10:21:22.570297 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55986 (local address 127.0.206.124:45531)
0619 10:21:22.570332 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0619 10:21:22.570334 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 10:21:22.570551 (+ 217us) server_negotiation.cc:407] Connection header received
0619 10:21:22.570680 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:21:22.570681 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:21:22.570691 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:21:22.570718 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:21:22.571046 (+ 328us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55986: BlockingRecv error: recv got EOF from 127.0.0.1:55986 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
I20260619 10:21:22.591372 23114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:21:22.588377 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40130 (local address 127.0.206.125:43989)
0619 10:21:22.588424 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0619 10:21:22.588426 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 10:21:22.588528 (+ 102us) server_negotiation.cc:407] Connection header received
0619 10:21:22.588581 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:21:22.588582 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:21:22.588593 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:21:22.588651 (+ 58us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 10:21:22.588794 (+ 143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:21:22.589508 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:21:22.590634 (+ 1126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:21:22.590911 (+ 277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:21:22.590989 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:21:22.591132 (+ 143us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 10:21:22.591167 (+ 35us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 10:21:22.591225 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0619 10:21:22.591282 (+ 57us) server_negotiation.cc:299] Negotiation successful
0619 10:21:22.591300 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
W20260619 10:21:22.593657 23188 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 10:21:22.593039 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55998 (local address 127.0.206.124:45531)
0619 10:21:22.593078 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0619 10:21:22.593080 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 10:21:22.593157 (+ 77us) server_negotiation.cc:407] Connection header received
0619 10:21:22.593215 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:21:22.593216 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:21:22.593227 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:21:22.593290 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:21:22.593607 (+ 317us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55998: BlockingRecv error: recv got EOF from 127.0.0.1:55998 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
I20260619 10:21:22.629436 23114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:21:22.626773 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40146 (local address 127.0.206.125:43989)
0619 10:21:22.626848 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0619 10:21:22.626850 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 10:21:22.626946 (+ 96us) server_negotiation.cc:407] Connection header received
0619 10:21:22.626998 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:21:22.626999 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:21:22.627010 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:21:22.627093 (+ 83us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 10:21:22.627213 (+ 120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:21:22.627962 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:21:22.628711 (+ 749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:21:22.629001 (+ 290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:21:22.629082 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:21:22.629202 (+ 120us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 10:21:22.629237 (+ 35us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 10:21:22.629296 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0619 10:21:22.629338 (+ 42us) server_negotiation.cc:299] Negotiation successful
0619 10:21:22.629357 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
W20260619 10:21:22.631845 23188 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 10:21:22.631183 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56004 (local address 127.0.206.124:45531)
0619 10:21:22.631240 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0619 10:21:22.631242 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 10:21:22.631302 (+ 60us) server_negotiation.cc:407] Connection header received
0619 10:21:22.631375 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:21:22.631375 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:21:22.631384 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:21:22.631428 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:21:22.631780 (+ 352us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56004: BlockingRecv error: recv got EOF from 127.0.0.1:56004 (error 108)
Metrics: {"server-negotiator.queue_time_us":31}
I20260619 10:21:22.699365 23114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:21:22.696857 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40148 (local address 127.0.206.125:43989)
0619 10:21:22.696932 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0619 10:21:22.696934 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 10:21:22.697094 (+ 160us) server_negotiation.cc:407] Connection header received
0619 10:21:22.697165 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:21:22.697165 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:21:22.697174 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:21:22.697224 (+ 50us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 10:21:22.697353 (+ 129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:21:22.697845 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:21:22.698658 (+ 813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:21:22.698845 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:21:22.698895 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:21:22.699012 (+ 117us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 10:21:22.699042 (+ 30us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 10:21:22.699077 (+ 35us) server_negotiation.cc:1035] Waiting for connection context
0619 10:21:22.699244 (+ 167us) server_negotiation.cc:299] Negotiation successful
0619 10:21:22.699274 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
W20260619 10:21:22.701443 23188 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 10:21:22.700945 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56016 (local address 127.0.206.124:45531)
0619 10:21:22.700998 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0619 10:21:22.701000 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 10:21:22.701008 (+ 8us) server_negotiation.cc:407] Connection header received
0619 10:21:22.701059 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:21:22.701059 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:21:22.701068 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:21:22.701111 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:21:22.701398 (+ 287us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56016: BlockingRecv error: recv got EOF from 127.0.0.1:56016 (error 108)
Metrics: {"server-negotiator.queue_time_us":25}
I20260619 10:21:22.833097 23114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:21:22.830703 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40156 (local address 127.0.206.125:43989)
0619 10:21:22.830772 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0619 10:21:22.830774 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 10:21:22.830888 (+ 114us) server_negotiation.cc:407] Connection header received
0619 10:21:22.830964 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:21:22.830965 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:21:22.830976 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:21:22.831062 (+ 86us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 10:21:22.831199 (+ 137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:21:22.831675 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:21:22.832425 (+ 750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:21:22.832608 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:21:22.832683 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:21:22.832827 (+ 144us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 10:21:22.832864 (+ 37us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 10:21:22.832923 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0619 10:21:22.832977 (+ 54us) server_negotiation.cc:299] Negotiation successful
0619 10:21:22.832995 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
W20260619 10:21:22.835392 23188 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 10:21:22.834773 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56032 (local address 127.0.206.124:45531)
0619 10:21:22.834841 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0619 10:21:22.834843 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 10:21:22.834917 (+ 74us) server_negotiation.cc:407] Connection header received
0619 10:21:22.834983 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:21:22.834983 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:21:22.834994 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:21:22.835041 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:21:22.835327 (+ 286us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56032: BlockingRecv error: recv got EOF from 127.0.0.1:56032 (error 108)
Metrics: {"server-negotiator.queue_time_us":37}
I20260619 10:21:23.095077 23114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:21:23.092577 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40172 (local address 127.0.206.125:43989)
0619 10:21:23.092658 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0619 10:21:23.092660 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 10:21:23.092672 (+ 12us) server_negotiation.cc:407] Connection header received
0619 10:21:23.092740 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:21:23.092741 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:21:23.092749 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:21:23.092810 (+ 61us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 10:21:23.092942 (+ 132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:21:23.093509 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:21:23.094299 (+ 790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:21:23.094578 (+ 279us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:21:23.094678 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:21:23.094825 (+ 147us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 10:21:23.094861 (+ 36us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 10:21:23.094910 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0619 10:21:23.094999 (+ 89us) server_negotiation.cc:299] Negotiation successful
0619 10:21:23.095015 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
W20260619 10:21:23.097163 23188 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 10:21:23.096662 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56040 (local address 127.0.206.124:45531)
0619 10:21:23.096701 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0619 10:21:23.096702 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 10:21:23.096708 (+ 6us) server_negotiation.cc:407] Connection header received
0619 10:21:23.096733 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:21:23.096733 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:21:23.096742 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:21:23.096789 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:21:23.097084 (+ 295us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56040: BlockingRecv error: recv got EOF from 127.0.0.1:56040 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
I20260619 10:21:23.400895 23202 catalog_manager.cc:1259] Loaded cluster ID: 87fb1ca0c32446d8b047d2943ec53298
I20260619 10:21:23.400996 23202 catalog_manager.cc:1552] T 00000000000000000000000000000000 P cd257ada5d4a47c4a69e0d5c1d532ff0: loading cluster ID for follower catalog manager: success
I20260619 10:21:23.401784 23202 catalog_manager.cc:1574] T 00000000000000000000000000000000 P cd257ada5d4a47c4a69e0d5c1d532ff0: acquiring CA information for follower catalog manager: success
I20260619 10:21:23.402125 23202 catalog_manager.cc:1602] T 00000000000000000000000000000000 P cd257ada5d4a47c4a69e0d5c1d532ff0: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260619 10:21:23.612442 23369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:21:23.610201 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40174 (local address 127.0.206.125:43989)
0619 10:21:23.610317 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0619 10:21:23.610319 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 10:21:23.610363 (+ 44us) server_negotiation.cc:407] Connection header received
0619 10:21:23.610435 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:21:23.610436 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:21:23.610450 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:21:23.610531 (+ 81us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 10:21:23.610696 (+ 165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:21:23.611184 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:21:23.611884 (+ 700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:21:23.612078 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:21:23.612130 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:21:23.612246 (+ 116us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 10:21:23.612274 (+ 28us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 10:21:23.612312 (+ 38us) server_negotiation.cc:1035] Waiting for connection context
0619 10:21:23.612361 (+ 49us) server_negotiation.cc:299] Negotiation successful
0619 10:21:23.612378 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":90,"thread_start_us":68,"threads_started":1}
I20260619 10:21:23.616459 23375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:21:23.614060 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56042 (local address 127.0.206.124:45531)
0619 10:21:23.614163 (+ 103us) server_negotiation.cc:206] Beginning negotiation
0619 10:21:23.614165 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 10:21:23.614174 (+ 9us) server_negotiation.cc:407] Connection header received
0619 10:21:23.614199 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:21:23.614199 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:21:23.614272 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:21:23.614347 (+ 75us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 10:21:23.614465 (+ 118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:21:23.614945 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:21:23.615666 (+ 721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:21:23.615882 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:21:23.615956 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:21:23.616125 (+ 169us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 10:21:23.616240 (+ 115us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 10:21:23.616326 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0619 10:21:23.616349 (+ 23us) server_negotiation.cc:299] Negotiation successful
0619 10:21:23.616370 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81,"thread_start_us":53,"threads_started":1}
I20260619 10:21:23.620780 23369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:21:23.618072 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40184 (local address 127.0.206.125:43989)
0619 10:21:23.618170 (+ 98us) server_negotiation.cc:206] Beginning negotiation
0619 10:21:23.618172 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 10:21:23.618179 (+ 7us) server_negotiation.cc:407] Connection header received
0619 10:21:23.618204 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:21:23.618205 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:21:23.618216 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:21:23.618263 (+ 47us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 10:21:23.618424 (+ 161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:21:23.619176 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:21:23.620025 (+ 849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:21:23.620335 (+ 310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:21:23.620404 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:21:23.620530 (+ 126us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 10:21:23.620560 (+ 30us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 10:21:23.620606 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0619 10:21:23.620688 (+ 82us) server_negotiation.cc:299] Negotiation successful
0619 10:21:23.620706 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260619 10:21:23.621127 23375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:21:23.618121 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56048 (local address 127.0.206.124:45531)
0619 10:21:23.618162 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0619 10:21:23.618164 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 10:21:23.618253 (+ 89us) server_negotiation.cc:407] Connection header received
0619 10:21:23.618300 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:21:23.618301 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:21:23.618312 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:21:23.618364 (+ 52us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 10:21:23.618456 (+ 92us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:21:23.619127 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:21:23.620098 (+ 971us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:21:23.620542 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:21:23.620603 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:21:23.620825 (+ 222us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 10:21:23.620857 (+ 32us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 10:21:23.620900 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0619 10:21:23.621030 (+ 130us) server_negotiation.cc:299] Negotiation successful
0619 10:21:23.621048 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260619 10:21:23.621349 23382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:21:23.617981 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47238 (local address 127.0.206.126:35571)
0619 10:21:23.618216 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0619 10:21:23.618218 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 10:21:23.618376 (+ 158us) server_negotiation.cc:407] Connection header received
0619 10:21:23.618420 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:21:23.618421 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:21:23.618488 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:21:23.618566 (+ 78us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 10:21:23.618716 (+ 150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:21:23.619511 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:21:23.620392 (+ 881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:21:23.620665 (+ 273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:21:23.620726 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:21:23.620934 (+ 208us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 10:21:23.621048 (+ 114us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 10:21:23.621102 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0619 10:21:23.621247 (+ 145us) server_negotiation.cc:299] Negotiation successful
0619 10:21:23.621275 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":217,"thread_start_us":48,"threads_started":1}
I20260619 10:21:23.622879 825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTU9OP6/build/release/bin/kudu with pid 22967
I20260619 10:21:23.627825 825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTU9OP6/build/release/bin/kudu with pid 23042
I20260619 10:21:23.632648 825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTU9OP6/build/release/bin/kudu with pid 23117
2026-06-19T10:21:23Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (2819 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781864224577058-825-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 10:21:23 dist-test-slave-6pt2 krb5kdc[23386](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 10:21:23 dist-test-slave-6pt2 krb5kdc[23386](info): set up 2 sockets
Jun 19 10:21:23 dist-test-slave-6pt2 krb5kdc[23386](info): commencing operation
krb5kdc: starting...
W20260619 10:21:24.650800 825 mini_kdc.cc:121] Time spent starting KDC: real 0.999s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781864224577058-825-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781864224577058-825-0/krb5kdc/test-user.keytab.
Jun 19 10:21:24 dist-test-slave-6pt2 krb5kdc[23386](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864484, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T10:21:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T10:21:24Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.206.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.206.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.206.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781864224577058-825-0/krb5kdc/kudu_127.0.206.126.keytab.
Entry for principal kudu/127.0.206.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781864224577058-825-0/krb5kdc/kudu_127.0.206.126.keytab.
I20260619 10:21:24.760905 825 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTU9OP6/build/release/bin/kudu
/tmp/dist-test-taskTU9OP6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781864224577058-825-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781864224577058-825-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781864224577058-825-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781864224577058-825-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.206.126:44269
--webserver_interface=127.0.206.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781864224577058-825-0/krb5kdc/kudu_127.0.206.126.keytab
--principal=kudu/127.0.206.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.206.84:42169
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.206.126:44269,127.0.206.125:38779,127.0.206.124:40941
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781864224577058-825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781864224577058-825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781864224577058-825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 10:21:24.838276 23399 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260619 10:21:24.838517 23399 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 10:21:24.838548 23399 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 10:21:24.838588 23399 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 10:21:24.839727 23399 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 10:21:24.839797 23399 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 10:21:24.839819 23399 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 10:21:24.839847 23399 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 10:21:24.839879 23399 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 10:21:24.841276 23399 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.206.84:42169
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781864224577058-825-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781864224577058-825-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.206.126:44269,127.0.206.125:38779,127.0.206.124:40941
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781864224577058-825-0/krb5kdc/kudu_127.0.206.126.keytab
--principal=kudu/127.0.206.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.206.126:44269
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781864224577058-825-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.206.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23399
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781864224577058-825-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 09:32:44 UTC on ccycloud.quasar-gbtldp.root.comops.site
build id 4455440
I20260619 10:21:24.841539 23399 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 10:21:24.841748 23399 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 10:21:24.846402 23407 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 10:21:24.846477 23399 server_base.cc:1034] running on GCE node
W20260619 10:21:24.846395 23405 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 10:21:24.846426 23404 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 10:21:24.846844 23399 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 10:21:24.847124 23399 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 10:21:24.848274 23399 hybrid_clock.cc:648] HybridClock initialized: now 1781864484848264 us; error 33 us; skew 500 ppm
Jun 19 10:21:24 dist-test-slave-6pt2 krb5kdc[23386](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864484, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 10:21:24.854445 23399 init.cc:377] Logged in from keytab as kudu/127.0.206.126@KRBTEST.COM (short username kudu)
I20260619 10:21:24.854902 23399 webserver.cc:460] Webserver started at http://127.0.206.126:34769/ using document root <none> and password file <none>
I20260619 10:21:24.855115 23399 fs_manager.cc:359] Metadata directory not provided
I20260619 10:21:24.855158 23399 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 10:21:24.855298 23399 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 10:21:24.856276 23399 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781864224577058-825-0/minicluster-data/master-0/data/instance:
uuid: "b3c64103572c4118908cc0f1f778b69a"
format_stamp: "Formatted at 2026-06-19 10:21:24 on dist-test-slave-6pt2"
server_key: "4a40c023acc833e5665887e222487983"
server_key_iv: "45948f1cb30f88a55b927c330529072b"
server_key_version: "encryptionkey@0"
I20260619 10:21:24.856621 23399 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781864224577058-825-0/minicluster-data/master-0/wal/instance:
uuid: "b3c64103572c4118908cc0f1f778b69a"
format_stamp: "Formatted at 2026-06-19 10:21:24 on dist-test-slave-6pt2"
server_key: "4a40c023acc833e5665887e222487983"
server_key_iv: "45948f1cb30f88a55b927c330529072b"
server_key_version: "encryptionkey@0"
I20260619 10:21:24.857874 23399 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260619 10:21:24.858532 23414 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 10:21:24.858727 23399 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260619 10:21:24.858821 23399 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781864224577058-825-0/minicluster-data/master-0/data,/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781864224577058-825-0/minicluster-data/master-0/wal
uuid: "b3c64103572c4118908cc0f1f778b69a"
format_stamp: "Formatted at 2026-06-19 10:21:24 on dist-test-slave-6pt2"
server_key: "4a40c023acc833e5665887e222487983"
server_key_iv: "45948f1cb30f88a55b927c330529072b"
server_key_version: "encryptionkey@0"
I20260619 10:21:24.858896 23399 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781864224577058-825-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781864224577058-825-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781864224577058-825-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 10:21:24.869004 23399 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 10:21:24.871156 23399 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 10:21:24.871309 23399 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 10:21:24.875021 23399 rpc_server.cc:307] RPC server started. Bound to: 127.0.206.126:44269
I20260619 10:21:24.875061 23466 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.206.126:44269 every 8 connection(s)
I20260619 10:21:24.875391 23399 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781864224577058-825-0/minicluster-data/master-0/data/info.pb
I20260619 10:21:24.876250 23467 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 10:21:24.877604 23467 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.206.126" port: 44269 } has no permanent_uuid. Determining permanent_uuid...
Jun 19 10:21:24 dist-test-slave-6pt2 krb5kdc[23386](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864484, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.126@KRBTEST.COM for kudu/127.0.206.126@KRBTEST.COM
I20260619 10:21:24.884796 825 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTU9OP6/build/release/bin/kudu as pid 23399
I20260619 10:21:24.884898 825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781864224577058-825-0/minicluster-data/master-0/wal/instance
I20260619 10:21:24.885090 825 external_mini_cluster.cc:1428] Setting key 606aea0986e219cf4c72adc8086253a9
I20260619 10:21:24.886745 23469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:21:24.878117 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:44269 (local address 127.0.0.1:43258)
0619 10:21:24.878312 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0619 10:21:24.878350 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0619 10:21:24.878862 (+ 512us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 10:21:24.879163 (+ 301us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 10:21:24.879165 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 10:21:24.879458 (+ 293us) client_negotiation.cc:189] Negotiated authn=SASL
0619 10:21:24.879601 (+ 143us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:21:24.879604 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:21:24.880597 (+ 993us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:21:24.880598 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:21:24.881234 (+ 636us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:21:24.881236 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:21:24.881724 (+ 488us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:21:24.881725 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:21:24.881786 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:21:24.882254 (+ 468us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 10:21:24.882261 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0619 10:21:24.883530 (+ 1269us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 10:21:24.885237 (+ 1707us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 10:21:24.885244 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 10:21:24.885246 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 10:21:24.885495 (+ 249us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 10:21:24.885714 (+ 219us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 10:21:24.885715 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 10:21:24.885715 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 10:21:24.885820 (+ 105us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 10:21:24.886139 (+ 319us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 10:21:24.886145 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 10:21:24.886215 (+ 70us) client_negotiation.cc:769] Sending connection context
0619 10:21:24.886309 (+ 94us) client_negotiation.cc:240] Negotiation successful
0619 10:21:24.886502 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":146,"mutex_wait_us":44,"thread_start_us":83,"threads_started":1}
I20260619 10:21:24.886746 23470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:21:24.878179 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43258 (local address 127.0.206.126:44269)
0619 10:21:24.878321 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0619 10:21:24.878329 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 10:21:24.878424 (+ 95us) server_negotiation.cc:407] Connection header received
0619 10:21:24.879045 (+ 621us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:21:24.879047 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:21:24.879063 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:21:24.879159 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:21:24.879702 (+ 543us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:21:24.880489 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:21:24.881337 (+ 848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:21:24.881602 (+ 265us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:21:24.881723 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:21:24.883609 (+ 1886us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 10:21:24.883615 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 10:21:24.883615 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 10:21:24.883621 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 10:21:24.885161 (+ 1540us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:21:24.885582 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:21:24.885584 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:21:24.885584 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:21:24.885651 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:21:24.885878 (+ 227us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:21:24.885879 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:21:24.885879 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:21:24.886074 (+ 195us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 10:21:24.886132 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0619 10:21:24.886342 (+ 210us) server_negotiation.cc:299] Negotiation successful
0619 10:21:24.886424 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":103,"thread_start_us":58,"threads_started":1}
I20260619 10:21:24.887449 23467 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.206.125" port: 38779 } has no permanent_uuid. Determining permanent_uuid...
I20260619 10:21:24.887748 23469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:21:24.887640 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.125:38779 (local address 127.0.0.1:36138)
0619 10:21:24.887684 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0619 10:21:24.887725 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.125:38779: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260619 10:21:24.887900 23416 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.206.125:38779: connect: Connection refused (error 111)
I20260619 10:21:24.888271 23469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:21:24.888185 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.125:38779 (local address 127.0.0.1:36144)
0619 10:21:24.888228 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0619 10:21:24.888253 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.125:38779: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260619 10:21:24.888376 23467 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.206.125:38779: Network error: Client connection negotiation failed: client connection to 127.0.206.125:38779: connect: Connection refused (error 111)
WARNING: no policy specified for kudu/127.0.206.125@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.206.125@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.206.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781864224577058-825-0/krb5kdc/kudu_127.0.206.125.keytab.
Entry for principal kudu/127.0.206.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781864224577058-825-0/krb5kdc/kudu_127.0.206.125.keytab.
I20260619 10:21:24.908689 825 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTU9OP6/build/release/bin/kudu
/tmp/dist-test-taskTU9OP6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781864224577058-825-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781864224577058-825-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781864224577058-825-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781864224577058-825-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.206.125:38779
--webserver_interface=127.0.206.125
--webserver_port=0
--keytab_file=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781864224577058-825-0/krb5kdc/kudu_127.0.206.125.keytab
--principal=kudu/127.0.206.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.206.84:42169
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.206.126:44269,127.0.206.125:38779,127.0.206.124:40941
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781864224577058-825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781864224577058-825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781864224577058-825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 10:21:24.937609 23467 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.206.125" port: 38779 } attempt: 1
I20260619 10:21:24.938091 23469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:21:24.937951 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.125:38779 (local address 127.0.0.1:36154)
0619 10:21:24.938017 (+ 66us) negotiation.cc:107] Waiting for socket to connect
0619 10:21:24.938060 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.125:38779: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
I20260619 10:21:24.938491 23469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:21:24.938406 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.125:38779 (local address 127.0.0.1:36156)
0619 10:21:24.938448 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0619 10:21:24.938473 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.125:38779: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260619 10:21:24.938616 23467 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.206.125:38779: Network error: Client connection negotiation failed: client connection to 127.0.206.125:38779: connect: Connection refused (error 111)
W20260619 10:21:24.986749 23474 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260619 10:21:24.986929 23474 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 10:21:24.986946 23474 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 10:21:24.986992 23474 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 10:21:24.988065 23474 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 10:21:24.988101 23474 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 10:21:24.988114 23474 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 10:21:24.988163 23474 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 10:21:24.988181 23474 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 10:21:24.989565 23474 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.206.84:42169
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781864224577058-825-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781864224577058-825-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.0.206.126:44269,127.0.206.125:38779,127.0.206.124:40941
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781864224577058-825-0/krb5kdc/kudu_127.0.206.125.keytab
--principal=kudu/127.0.206.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.206.125:38779
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781864224577058-825-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.0.206.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23474
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781864224577058-825-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 09:32:44 UTC on ccycloud.quasar-gbtldp.root.comops.site
build id 4455440
I20260619 10:21:24.989795 23474 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 10:21:24.990028 23474 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 10:21:24.994495 23479 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 10:21:24.994532 23482 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 10:21:24.994585 23474 server_base.cc:1034] running on GCE node
W20260619 10:21:24.994504 23480 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 10:21:24.994894 23474 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 10:21:24.995193 23474 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 10:21:24.996341 23474 hybrid_clock.cc:648] HybridClock initialized: now 1781864484996323 us; error 39 us; skew 500 ppm
Jun 19 10:21:24 dist-test-slave-6pt2 krb5kdc[23386](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864484, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 10:21:24.998723 23474 init.cc:377] Logged in from keytab as kudu/127.0.206.125@KRBTEST.COM (short username kudu)
I20260619 10:21:24.999122 23474 webserver.cc:460] Webserver started at http://127.0.206.125:39521/ using document root <none> and password file <none>
I20260619 10:21:24.999339 23474 fs_manager.cc:359] Metadata directory not provided
I20260619 10:21:24.999379 23474 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 10:21:24.999503 23474 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 10:21:25.000514 23474 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781864224577058-825-0/minicluster-data/master-1/data/instance:
uuid: "cbfcf8b295f544d0802cee914e31c384"
format_stamp: "Formatted at 2026-06-19 10:21:24 on dist-test-slave-6pt2"
server_key: "93420f00751153348804884d356e6b08"
server_key_iv: "aade31510a6b40255f3ad612ac5f76a6"
server_key_version: "encryptionkey@0"
I20260619 10:21:25.000859 23474 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781864224577058-825-0/minicluster-data/master-1/wal/instance:
uuid: "cbfcf8b295f544d0802cee914e31c384"
format_stamp: "Formatted at 2026-06-19 10:21:24 on dist-test-slave-6pt2"
server_key: "93420f00751153348804884d356e6b08"
server_key_iv: "aade31510a6b40255f3ad612ac5f76a6"
server_key_version: "encryptionkey@0"
I20260619 10:21:25.002115 23474 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260619 10:21:25.002874 23489 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 10:21:25.003037 23474 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 10:21:25.003113 23474 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781864224577058-825-0/minicluster-data/master-1/data,/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781864224577058-825-0/minicluster-data/master-1/wal
uuid: "cbfcf8b295f544d0802cee914e31c384"
format_stamp: "Formatted at 2026-06-19 10:21:24 on dist-test-slave-6pt2"
server_key: "93420f00751153348804884d356e6b08"
server_key_iv: "aade31510a6b40255f3ad612ac5f76a6"
server_key_version: "encryptionkey@0"
I20260619 10:21:25.003162 23474 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781864224577058-825-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781864224577058-825-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781864224577058-825-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 10:21:25.006754 23467 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.206.125" port: 38779 } attempt: 2
I20260619 10:21:25.007103 23469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:21:25.007012 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.125:38779 (local address 127.0.0.1:36168)
0619 10:21:25.007052 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0619 10:21:25.007080 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.125:38779: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":23}
I20260619 10:21:25.007390 23469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:21:25.007340 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.125:38779 (local address 127.0.0.1:36170)
0619 10:21:25.007359 (+ 19us) negotiation.cc:107] Waiting for socket to connect
0619 10:21:25.007377 (+ 18us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.125:38779: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":10}
W20260619 10:21:25.007472 23467 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.206.125:38779: Network error: Client connection negotiation failed: client connection to 127.0.206.125:38779: connect: Connection refused (error 111)
I20260619 10:21:25.017994 23474 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 10:21:25.020103 23474 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 10:21:25.020201 23474 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 10:21:25.023788 23474 rpc_server.cc:307] RPC server started. Bound to: 127.0.206.125:38779
I20260619 10:21:25.023818 23541 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.206.125:38779 every 8 connection(s)
I20260619 10:21:25.024144 23474 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781864224577058-825-0/minicluster-data/master-1/data/info.pb
I20260619 10:21:25.025045 23542 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 10:21:25.026425 23542 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.206.126" port: 44269 } has no permanent_uuid. Determining permanent_uuid...
Jun 19 10:21:25 dist-test-slave-6pt2 krb5kdc[23386](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864484, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.125@KRBTEST.COM for kudu/127.0.206.126@KRBTEST.COM
I20260619 10:21:25.032951 825 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTU9OP6/build/release/bin/kudu as pid 23474
I20260619 10:21:25.033056 825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781864224577058-825-0/minicluster-data/master-1/wal/instance
I20260619 10:21:25.033318 825 external_mini_cluster.cc:1428] Setting key b968252a5f3b791ea22ea2671f444122
I20260619 10:21:25.034505 23544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:21:25.027058 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:44269 (local address 127.0.0.1:43272)
0619 10:21:25.027253 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0619 10:21:25.027283 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0619 10:21:25.027691 (+ 408us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 10:21:25.027887 (+ 196us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 10:21:25.027890 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 10:21:25.028101 (+ 211us) client_negotiation.cc:189] Negotiated authn=SASL
0619 10:21:25.028296 (+ 195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:21:25.028300 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:21:25.028871 (+ 571us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:21:25.028872 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:21:25.029551 (+ 679us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:21:25.029552 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:21:25.029857 (+ 305us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:21:25.029858 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:21:25.029974 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:21:25.030419 (+ 445us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 10:21:25.030426 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0619 10:21:25.031744 (+ 1318us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 10:21:25.033235 (+ 1491us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 10:21:25.033241 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 10:21:25.033242 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 10:21:25.033518 (+ 276us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 10:21:25.033716 (+ 198us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 10:21:25.033716 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 10:21:25.033717 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 10:21:25.033806 (+ 89us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 10:21:25.034079 (+ 273us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 10:21:25.034083 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 10:21:25.034181 (+ 98us) client_negotiation.cc:769] Sending connection context
0619 10:21:25.034256 (+ 75us) client_negotiation.cc:240] Negotiation successful
0619 10:21:25.034360 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":161,"thread_start_us":88,"threads_started":1}
I20260619 10:21:25.034525 23470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:21:25.027062 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43272 (local address 127.0.206.126:44269)
0619 10:21:25.027129 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0619 10:21:25.027131 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 10:21:25.027359 (+ 228us) server_negotiation.cc:407] Connection header received
0619 10:21:25.027765 (+ 406us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:21:25.027766 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:21:25.027776 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:21:25.027824 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:21:25.028361 (+ 537us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:21:25.028801 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:21:25.029614 (+ 813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:21:25.029776 (+ 162us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:21:25.029819 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:21:25.031823 (+ 2004us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 10:21:25.031824 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 10:21:25.031824 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 10:21:25.031843 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0619 10:21:25.033157 (+ 1314us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:21:25.033592 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:21:25.033593 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:21:25.033594 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:21:25.033652 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:21:25.033868 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:21:25.033869 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:21:25.033870 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:21:25.034010 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 10:21:25.034086 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0619 10:21:25.034299 (+ 213us) server_negotiation.cc:299] Negotiation successful
0619 10:21:25.034388 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260619 10:21:25.035120 23542 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.206.125" port: 38779 } has no permanent_uuid. Determining permanent_uuid...
Jun 19 10:21:25 dist-test-slave-6pt2 krb5kdc[23386](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864484, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.125@KRBTEST.COM for kudu/127.0.206.125@KRBTEST.COM
WARNING: no policy specified for kudu/127.0.206.124@KRBTEST.COM; defaulting to no policy
I20260619 10:21:25.041854 23544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:21:25.035349 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.125:38779 (local address 127.0.0.1:36174)
0619 10:21:25.035395 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0619 10:21:25.035405 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 10:21:25.035448 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 10:21:25.035681 (+ 233us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 10:21:25.035682 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 10:21:25.035803 (+ 121us) client_negotiation.cc:189] Negotiated authn=SASL
0619 10:21:25.035841 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:21:25.035842 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:21:25.036661 (+ 819us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:21:25.036662 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:21:25.037164 (+ 502us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:21:25.037166 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:21:25.037671 (+ 505us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:21:25.037672 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:21:25.037736 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:21:25.038114 (+ 378us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 10:21:25.038119 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 10:21:25.039261 (+ 1142us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 10:21:25.040853 (+ 1592us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 10:21:25.040854 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 10:21:25.040854 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 10:21:25.041004 (+ 150us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 10:21:25.041208 (+ 204us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 10:21:25.041209 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 10:21:25.041209 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 10:21:25.041256 (+ 47us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 10:21:25.041550 (+ 294us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 10:21:25.041551 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 10:21:25.041588 (+ 37us) client_negotiation.cc:769] Sending connection context
0619 10:21:25.041658 (+ 70us) client_negotiation.cc:240] Negotiation successful
0619 10:21:25.041748 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":31}
I20260619 10:21:25.041945 23546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:21:25.035411 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36174 (local address 127.0.206.125:38779)
0619 10:21:25.035559 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0619 10:21:25.035566 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 10:21:25.035579 (+ 13us) server_negotiation.cc:407] Connection header received
0619 10:21:25.035594 (+ 15us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:21:25.035596 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:21:25.035605 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:21:25.035681 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:21:25.035913 (+ 232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:21:25.036569 (+ 656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:21:25.037250 (+ 681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:21:25.037594 (+ 344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:21:25.037675 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:21:25.039349 (+ 1674us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 10:21:25.039355 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 10:21:25.039355 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 10:21:25.039361 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 10:21:25.040763 (+ 1402us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:21:25.041086 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:21:25.041089 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:21:25.041089 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:21:25.041136 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:21:25.041320 (+ 184us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:21:25.041320 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:21:25.041321 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:21:25.041447 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 10:21:25.041569 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0619 10:21:25.041686 (+ 117us) server_negotiation.cc:299] Negotiation successful
0619 10:21:25.041814 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":49,"server-negotiator.queue_time_us":105,"thread_start_us":65,"threads_started":1}
I20260619 10:21:25.042533 23542 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.206.124" port: 40941 } has no permanent_uuid. Determining permanent_uuid...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.206.124@KRBTEST.COM" created.
I20260619 10:21:25.042874 23544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:21:25.042742 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.124:40941 (local address 127.0.0.1:35214)
0619 10:21:25.042796 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0619 10:21:25.042842 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.124:40941: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260619 10:21:25.043087 23490 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.206.124:40941: connect: Connection refused (error 111)
I20260619 10:21:25.043419 23544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:21:25.043360 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.124:40941 (local address 127.0.0.1:35224)
0619 10:21:25.043383 (+ 23us) negotiation.cc:107] Waiting for socket to connect
0619 10:21:25.043403 (+ 20us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.124:40941: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":13}
W20260619 10:21:25.043527 23542 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.206.124:40941: Network error: Client connection negotiation failed: client connection to 127.0.206.124:40941: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.206.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781864224577058-825-0/krb5kdc/kudu_127.0.206.124.keytab.
Entry for principal kudu/127.0.206.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781864224577058-825-0/krb5kdc/kudu_127.0.206.124.keytab.
I20260619 10:21:25.055231 825 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskTU9OP6/build/release/bin/kudu
/tmp/dist-test-taskTU9OP6/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781864224577058-825-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781864224577058-825-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781864224577058-825-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781864224577058-825-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.206.124:40941
--webserver_interface=127.0.206.124
--webserver_port=0
--keytab_file=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781864224577058-825-0/krb5kdc/kudu_127.0.206.124.keytab
--principal=kudu/127.0.206.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.206.84:42169
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.206.126:44269,127.0.206.125:38779,127.0.206.124:40941
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781864224577058-825-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781864224577058-825-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781864224577058-825-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 10:21:25.092718 23542 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.206.124" port: 40941 } attempt: 1
I20260619 10:21:25.093261 23544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:21:25.093123 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.124:40941 (local address 127.0.0.1:35236)
0619 10:21:25.093182 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0619 10:21:25.093219 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.124:40941: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260619 10:21:25.093616 23544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:21:25.093529 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.124:40941 (local address 127.0.0.1:35238)
0619 10:21:25.093571 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0619 10:21:25.093592 (+ 21us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.124:40941: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
W20260619 10:21:25.093690 23542 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.206.124:40941: Network error: Client connection negotiation failed: client connection to 127.0.206.124:40941: connect: Connection refused (error 111)
I20260619 10:21:25.098623 23467 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.206.125" port: 38779 } attempt: 3
Jun 19 10:21:25 dist-test-slave-6pt2 krb5kdc[23386](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864484, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.126@KRBTEST.COM for kudu/127.0.206.125@KRBTEST.COM
I20260619 10:21:25.105058 23469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:21:25.098940 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.125:38779 (local address 127.0.0.1:36190)
0619 10:21:25.098985 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0619 10:21:25.098995 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 10:21:25.099062 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 10:21:25.099189 (+ 127us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 10:21:25.099189 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 10:21:25.099341 (+ 152us) client_negotiation.cc:189] Negotiated authn=SASL
0619 10:21:25.099399 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:21:25.099400 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:21:25.100037 (+ 637us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:21:25.100038 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:21:25.100557 (+ 519us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:21:25.100559 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:21:25.100945 (+ 386us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:21:25.100946 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:21:25.101020 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:21:25.101451 (+ 431us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 10:21:25.101455 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 10:21:25.102586 (+ 1131us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 10:21:25.104259 (+ 1673us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 10:21:25.104259 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 10:21:25.104260 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 10:21:25.104406 (+ 146us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 10:21:25.104588 (+ 182us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 10:21:25.104588 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 10:21:25.104589 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 10:21:25.104633 (+ 44us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 10:21:25.104858 (+ 225us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 10:21:25.104858 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 10:21:25.104892 (+ 34us) client_negotiation.cc:769] Sending connection context
0619 10:21:25.104935 (+ 43us) client_negotiation.cc:240] Negotiation successful
0619 10:21:25.105002 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":26}
I20260619 10:21:25.105109 23546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:21:25.098998 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36190 (local address 127.0.206.125:38779)
0619 10:21:25.099054 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0619 10:21:25.099056 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 10:21:25.099066 (+ 10us) server_negotiation.cc:407] Connection header received
0619 10:21:25.099127 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:21:25.099128 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:21:25.099139 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:21:25.099186 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:21:25.099474 (+ 288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:21:25.099950 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:21:25.100638 (+ 688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:21:25.100868 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:21:25.100944 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:21:25.102715 (+ 1771us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 10:21:25.102717 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 10:21:25.102717 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 10:21:25.102725 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 10:21:25.104188 (+ 1463us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:21:25.104471 (+ 283us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:21:25.104473 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:21:25.104473 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:21:25.104529 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:21:25.104684 (+ 155us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:21:25.104684 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:21:25.104685 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:21:25.104803 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 10:21:25.104862 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0619 10:21:25.104956 (+ 94us) server_negotiation.cc:299] Negotiation successful
0619 10:21:25.105034 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260619 10:21:25.105469 23467 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.206.124" port: 40941 } has no permanent_uuid. Determining permanent_uuid...
I20260619 10:21:25.105792 23469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:21:25.105674 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.124:40941 (local address 127.0.0.1:35240)
0619 10:21:25.105720 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0619 10:21:25.105765 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.124:40941: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
I20260619 10:21:25.106088 23469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:21:25.106026 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.124:40941 (local address 127.0.0.1:35252)
0619 10:21:25.106056 (+ 30us) negotiation.cc:107] Waiting for socket to connect
0619 10:21:25.106075 (+ 19us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.124:40941: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":20}
W20260619 10:21:25.106187 23467 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.206.124:40941: Network error: Client connection negotiation failed: client connection to 127.0.206.124:40941: connect: Connection refused (error 111)
W20260619 10:21:25.133049 23549 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260619 10:21:25.133224 23549 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 10:21:25.133239 23549 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 10:21:25.133257 23549 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 10:21:25.134752 23549 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 10:21:25.134804 23549 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 10:21:25.134827 23549 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 10:21:25.134845 23549 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 10:21:25.134866 23549 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 10:21:25.136286 23549 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.206.84:42169
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781864224577058-825-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781864224577058-825-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.0.206.126:44269,127.0.206.125:38779,127.0.206.124:40941
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781864224577058-825-0/krb5kdc/kudu_127.0.206.124.keytab
--principal=kudu/127.0.206.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.206.124:40941
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781864224577058-825-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.0.206.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23549
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781864224577058-825-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 09:32:44 UTC on ccycloud.quasar-gbtldp.root.comops.site
build id 4455440
I20260619 10:21:25.136546 23549 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 10:21:25.136771 23549 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260619 10:21:25.137356 23467 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.206.124" port: 40941 } attempt: 1
I20260619 10:21:25.137817 23469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:21:25.137699 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.124:40941 (local address 127.0.0.1:35256)
0619 10:21:25.137756 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0619 10:21:25.137790 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.124:40941: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260619 10:21:25.138231 23469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:21:25.138116 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.124:40941 (local address 127.0.0.1:35258)
0619 10:21:25.138170 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0619 10:21:25.138201 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.124:40941: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260619 10:21:25.138326 23467 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.206.124:40941: Network error: Client connection negotiation failed: client connection to 127.0.206.124:40941: connect: Connection refused (error 111)
W20260619 10:21:25.141844 23557 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 10:21:25.141848 23554 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 10:21:25.141892 23549 server_base.cc:1034] running on GCE node
W20260619 10:21:25.141894 23555 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 10:21:25.142298 23549 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 10:21:25.142587 23549 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 10:21:25.143779 23549 hybrid_clock.cc:648] HybridClock initialized: now 1781864485143767 us; error 26 us; skew 500 ppm
Jun 19 10:21:25 dist-test-slave-6pt2 krb5kdc[23386](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864485, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 10:21:25.146131 23549 init.cc:377] Logged in from keytab as kudu/127.0.206.124@KRBTEST.COM (short username kudu)
I20260619 10:21:25.146535 23549 webserver.cc:460] Webserver started at http://127.0.206.124:33919/ using document root <none> and password file <none>
I20260619 10:21:25.146765 23549 fs_manager.cc:359] Metadata directory not provided
I20260619 10:21:25.146831 23549 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 10:21:25.146981 23549 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 10:21:25.147948 23549 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781864224577058-825-0/minicluster-data/master-2/data/instance:
uuid: "5277d3aff87d49a389c54ea34b837cf8"
format_stamp: "Formatted at 2026-06-19 10:21:25 on dist-test-slave-6pt2"
server_key: "1a766186ec5d46c71e8ff7efa07bbd3a"
server_key_iv: "9449ab16f219c70edd7f4ff7948dcb36"
server_key_version: "encryptionkey@0"
I20260619 10:21:25.148285 23549 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781864224577058-825-0/minicluster-data/master-2/wal/instance:
uuid: "5277d3aff87d49a389c54ea34b837cf8"
format_stamp: "Formatted at 2026-06-19 10:21:25 on dist-test-slave-6pt2"
server_key: "1a766186ec5d46c71e8ff7efa07bbd3a"
server_key_iv: "9449ab16f219c70edd7f4ff7948dcb36"
server_key_version: "encryptionkey@0"
I20260619 10:21:25.149533 23549 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.001s
I20260619 10:21:25.150218 23564 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 10:21:25.150394 23549 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 10:21:25.150487 23549 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781864224577058-825-0/minicluster-data/master-2/data,/tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781864224577058-825-0/minicluster-data/master-2/wal
uuid: "5277d3aff87d49a389c54ea34b837cf8"
format_stamp: "Formatted at 2026-06-19 10:21:25 on dist-test-slave-6pt2"
server_key: "1a766186ec5d46c71e8ff7efa07bbd3a"
server_key_iv: "9449ab16f219c70edd7f4ff7948dcb36"
server_key_version: "encryptionkey@0"
I20260619 10:21:25.150566 23549 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781864224577058-825-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781864224577058-825-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781864224577058-825-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 10:21:25.161844 23542 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.206.124" port: 40941 } attempt: 2
I20260619 10:21:25.162182 23549 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 10:21:25.162209 23544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:21:25.162096 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.124:40941 (local address 127.0.0.1:35262)
0619 10:21:25.162151 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0619 10:21:25.162185 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.124:40941: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
I20260619 10:21:25.162551 23544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:21:25.162436 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.124:40941 (local address 127.0.0.1:35272)
0619 10:21:25.162504 (+ 68us) negotiation.cc:107] Waiting for socket to connect
0619 10:21:25.162533 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.206.124:40941: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
W20260619 10:21:25.162693 23542 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.0.206.124:40941: Network error: Client connection negotiation failed: client connection to 127.0.206.124:40941: connect: Connection refused (error 111)
I20260619 10:21:25.164378 23549 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 10:21:25.164485 23549 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 10:21:25.168141 23549 rpc_server.cc:307] RPC server started. Bound to: 127.0.206.124:40941
I20260619 10:21:25.168206 23616 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.206.124:40941 every 8 connection(s)
I20260619 10:21:25.168540 23549 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781864224577058-825-0/minicluster-data/master-2/data/info.pb
I20260619 10:21:25.168988 825 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskTU9OP6/build/release/bin/kudu as pid 23549
I20260619 10:21:25.169096 825 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskTU9OP6/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781864224577058-825-0/minicluster-data/master-2/wal/instance
I20260619 10:21:25.169286 825 external_mini_cluster.cc:1428] Setting key 305c4bacc6776ced34a5ddc58a519710
I20260619 10:21:25.169620 23617 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 10:21:25.171278 23617 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.206.126" port: 44269 } has no permanent_uuid. Determining permanent_uuid...
Jun 19 10:21:25 dist-test-slave-6pt2 krb5kdc[23386](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864485, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.124@KRBTEST.COM for kudu/127.0.206.126@KRBTEST.COM
Jun 19 10:21:25 dist-test-slave-6pt2 krb5kdc[23386](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864484, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.206.124@KRBTEST.COM
I20260619 10:21:25.180390 23470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:21:25.172038 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43284 (local address 127.0.206.126:44269)
0619 10:21:25.172100 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0619 10:21:25.172102 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 10:21:25.172344 (+ 242us) server_negotiation.cc:407] Connection header received
0619 10:21:25.172404 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:21:25.172406 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:21:25.172415 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:21:25.172461 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:21:25.172945 (+ 484us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:21:25.173581 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:21:25.174667 (+ 1086us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:21:25.174958 (+ 291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:21:25.175048 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:21:25.177132 (+ 2084us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 10:21:25.177134 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 10:21:25.177134 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 10:21:25.177140 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 10:21:25.179186 (+ 2046us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:21:25.179535 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:21:25.179536 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:21:25.179537 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:21:25.179602 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:21:25.179825 (+ 223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:21:25.179826 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:21:25.179826 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:21:25.179935 (+ 109us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 10:21:25.180006 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0619 10:21:25.180213 (+ 207us) server_negotiation.cc:299] Negotiation successful
0619 10:21:25.180286 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260619 10:21:25.180727 23621 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:21:25.171955 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.126:44269 (local address 127.0.0.1:43284)
0619 10:21:25.172239 (+ 284us) negotiation.cc:107] Waiting for socket to connect
0619 10:21:25.172282 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0619 10:21:25.172341 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 10:21:25.172487 (+ 146us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 10:21:25.172490 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 10:21:25.172737 (+ 247us) client_negotiation.cc:189] Negotiated authn=SASL
0619 10:21:25.172825 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:21:25.172830 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:21:25.173690 (+ 860us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:21:25.173691 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:21:25.174553 (+ 862us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:21:25.174555 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:21:25.175069 (+ 514us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:21:25.175071 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:21:25.175179 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:21:25.175669 (+ 490us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 10:21:25.175677 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 10:21:25.177039 (+ 1362us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 10:21:25.179268 (+ 2229us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 10:21:25.179274 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 10:21:25.179277 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 10:21:25.179461 (+ 184us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 10:21:25.179678 (+ 217us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 10:21:25.179679 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 10:21:25.179680 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 10:21:25.179760 (+ 80us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 10:21:25.179999 (+ 239us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 10:21:25.180005 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 10:21:25.180098 (+ 93us) client_negotiation.cc:769] Sending connection context
0619 10:21:25.180193 (+ 95us) client_negotiation.cc:240] Negotiation successful
0619 10:21:25.180325 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":244,"thread_start_us":113,"threads_started":1}
I20260619 10:21:25.180728 23620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:21:25.169743 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35274 (local address 127.0.206.124:40941)
0619 10:21:25.169904 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0619 10:21:25.169912 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 10:21:25.170810 (+ 898us) server_negotiation.cc:407] Connection header received
0619 10:21:25.171210 (+ 400us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:21:25.171216 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:21:25.171259 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:21:25.171387 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:21:25.172194 (+ 807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:21:25.173146 (+ 952us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:21:25.174020 (+ 874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:21:25.174622 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:21:25.174718 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:21:25.177613 (+ 2895us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 10:21:25.177620 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 10:21:25.177620 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 10:21:25.177627 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 10:21:25.179213 (+ 1586us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:21:25.179605 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:21:25.179608 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:21:25.179608 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:21:25.179700 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:21:25.179899 (+ 199us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:21:25.179900 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:21:25.179900 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:21:25.180094 (+ 194us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 10:21:25.180228 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0619 10:21:25.180316 (+ 88us) server_negotiation.cc:299] Negotiation successful
0619 10:21:25.180409 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":117,"thread_start_us":68,"threads_started":1}
I20260619 10:21:25.181310 23617 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.206.125" port: 38779 } has no permanent_uuid. Determining permanent_uuid...
Jun 19 10:21:25 dist-test-slave-6pt2 krb5kdc[23386](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864485, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.124@KRBTEST.COM for kudu/127.0.206.125@KRBTEST.COM
Jun 19 10:21:25 dist-test-slave-6pt2 krb5kdc[23386](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864484, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.206.126@KRBTEST.COM
I20260619 10:21:25.188436 23546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:21:25.181618 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36200 (local address 127.0.206.125:38779)
0619 10:21:25.181700 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0619 10:21:25.181702 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 10:21:25.181708 (+ 6us) server_negotiation.cc:407] Connection header received
0619 10:21:25.181729 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:21:25.181730 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:21:25.181746 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:21:25.181807 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:21:25.182094 (+ 287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:21:25.182753 (+ 659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:21:25.183604 (+ 851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:21:25.183811 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:21:25.183871 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:21:25.185734 (+ 1863us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 10:21:25.185735 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 10:21:25.185736 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 10:21:25.185742 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 10:21:25.187333 (+ 1591us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:21:25.187682 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:21:25.187683 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:21:25.187683 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:21:25.187730 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:21:25.187929 (+ 199us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:21:25.187930 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:21:25.187930 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:21:25.188016 (+ 86us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 10:21:25.188090 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0619 10:21:25.188280 (+ 190us) server_negotiation.cc:299] Negotiation successful
0619 10:21:25.188350 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260619 10:21:25.188447 23621 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:21:25.181547 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.125:38779 (local address 127.0.0.1:36200)
0619 10:21:25.181578 (+ 31us) negotiation.cc:107] Waiting for socket to connect
0619 10:21:25.181588 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 10:21:25.181629 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 10:21:25.181810 (+ 181us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 10:21:25.181811 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 10:21:25.181964 (+ 153us) client_negotiation.cc:189] Negotiated authn=SASL
0619 10:21:25.182012 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:21:25.182014 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:21:25.182821 (+ 807us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:21:25.182822 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:21:25.183527 (+ 705us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:21:25.183530 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:21:25.183877 (+ 347us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:21:25.183878 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:21:25.184085 (+ 207us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:21:25.184412 (+ 327us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 10:21:25.184416 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 10:21:25.185632 (+ 1216us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 10:21:25.187410 (+ 1778us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 10:21:25.187411 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 10:21:25.187412 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 10:21:25.187612 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 10:21:25.187810 (+ 198us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 10:21:25.187811 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 10:21:25.187811 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 10:21:25.187872 (+ 61us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 10:21:25.188097 (+ 225us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 10:21:25.188098 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 10:21:25.188138 (+ 40us) client_negotiation.cc:769] Sending connection context
0619 10:21:25.188237 (+ 99us) client_negotiation.cc:240] Negotiation successful
0619 10:21:25.188350 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":17}
I20260619 10:21:25.189083 23470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:21:25.181663 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43294 (local address 127.0.206.126:44269)
0619 10:21:25.181852 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0619 10:21:25.181855 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 10:21:25.181861 (+ 6us) server_negotiation.cc:407] Connection header received
0619 10:21:25.181877 (+ 16us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:21:25.181878 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:21:25.181885 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:21:25.181937 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:21:25.182321 (+ 384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:21:25.182949 (+ 628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:21:25.183782 (+ 833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:21:25.184015 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:21:25.184062 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:21:25.186384 (+ 2322us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 10:21:25.186385 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 10:21:25.186385 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 10:21:25.186392 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 10:21:25.187997 (+ 1605us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:21:25.188340 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:21:25.188341 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:21:25.188342 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:21:25.188390 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:21:25.188616 (+ 226us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:21:25.188617 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:21:25.188617 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:21:25.188719 (+ 102us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 10:21:25.188866 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0619 10:21:25.188914 (+ 48us) server_negotiation.cc:299] Negotiation successful
0619 10:21:25.188979 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":173}
I20260619 10:21:25.189123 23617 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.206.124" port: 40941 } has no permanent_uuid. Determining permanent_uuid...
Jun 19 10:21:25 dist-test-slave-6pt2 krb5kdc[23386](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864485, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.124@KRBTEST.COM for kudu/127.0.206.124@KRBTEST.COM
Jun 19 10:21:25 dist-test-slave-6pt2 krb5kdc[23386](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864484, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.206.125@KRBTEST.COM
I20260619 10:21:25.195544 23621 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:21:25.189325 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.124:40941 (local address 127.0.0.1:35276)
0619 10:21:25.189466 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0619 10:21:25.189477 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 10:21:25.189519 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 10:21:25.189627 (+ 108us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 10:21:25.189627 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 10:21:25.189754 (+ 127us) client_negotiation.cc:189] Negotiated authn=SASL
0619 10:21:25.189774 (+ 20us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:21:25.189775 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:21:25.190393 (+ 618us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:21:25.190394 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:21:25.190934 (+ 540us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:21:25.190935 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:21:25.191307 (+ 372us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:21:25.191308 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:21:25.191355 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:21:25.191738 (+ 383us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 10:21:25.191743 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 10:21:25.193126 (+ 1383us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 10:21:25.194561 (+ 1435us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 10:21:25.194562 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 10:21:25.194563 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 10:21:25.194747 (+ 184us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 10:21:25.194948 (+ 201us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 10:21:25.194949 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 10:21:25.194949 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 10:21:25.194999 (+ 50us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 10:21:25.195230 (+ 231us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 10:21:25.195231 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 10:21:25.195264 (+ 33us) client_negotiation.cc:769] Sending connection context
0619 10:21:25.195329 (+ 65us) client_negotiation.cc:240] Negotiation successful
0619 10:21:25.195425 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":132}
I20260619 10:21:25.195590 23620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:21:25.189412 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35276 (local address 127.0.206.124:40941)
0619 10:21:25.189469 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0619 10:21:25.189470 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 10:21:25.189505 (+ 35us) server_negotiation.cc:407] Connection header received
0619 10:21:25.189571 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:21:25.189571 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:21:25.189580 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:21:25.189626 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:21:25.189846 (+ 220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:21:25.190311 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:21:25.191021 (+ 710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:21:25.191214 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:21:25.191306 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:21:25.193216 (+ 1910us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 10:21:25.193217 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 10:21:25.193217 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 10:21:25.193224 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 10:21:25.194487 (+ 1263us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:21:25.194828 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:21:25.194829 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:21:25.194829 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:21:25.194880 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:21:25.195073 (+ 193us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:21:25.195074 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:21:25.195075 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:21:25.195168 (+ 93us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 10:21:25.195236 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0619 10:21:25.195355 (+ 119us) server_negotiation.cc:299] Negotiation successful
0619 10:21:25.195471 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":60,"server-negotiator.queue_time_us":39}
I20260619 10:21:25.196906 23546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:21:25.189601 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36204 (local address 127.0.206.125:38779)
0619 10:21:25.189634 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0619 10:21:25.189636 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 10:21:25.189643 (+ 7us) server_negotiation.cc:407] Connection header received
0619 10:21:25.189664 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:21:25.189665 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:21:25.189673 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:21:25.189728 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:21:25.190179 (+ 451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:21:25.190693 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:21:25.191454 (+ 761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:21:25.191652 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:21:25.191717 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:21:25.194223 (+ 2506us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 10:21:25.194224 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 10:21:25.194224 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 10:21:25.194229 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0619 10:21:25.195509 (+ 1280us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:21:25.196199 (+ 690us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:21:25.196200 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:21:25.196200 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:21:25.196259 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:21:25.196440 (+ 181us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:21:25.196441 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:21:25.196441 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:21:25.196558 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 10:21:25.196703 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0619 10:21:25.196740 (+ 37us) server_negotiation.cc:299] Negotiation successful
0619 10:21:25.196808 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260619 10:21:25.197337 23617 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5277d3aff87d49a389c54ea34b837cf8: Bootstrap starting.
I20260619 10:21:25.198333 825 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 10:21:25.198359 23617 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5277d3aff87d49a389c54ea34b837cf8: Neither blocks nor log segments found. Creating new log.
I20260619 10:21:25.198661 23617 log.cc:824] T 00000000000000000000000000000000 P 5277d3aff87d49a389c54ea34b837cf8: Log is configured to *not* fsync() on all Append() calls
I20260619 10:21:25.199438 23617 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5277d3aff87d49a389c54ea34b837cf8: No bootstrap required, opened a new log
I20260619 10:21:25.199236 23441 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "b3c64103572c4118908cc0f1f778b69a"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43294
I20260619 10:21:25.200805 23617 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5277d3aff87d49a389c54ea34b837cf8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3c64103572c4118908cc0f1f778b69a" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 44269 } } peers { permanent_uuid: "cbfcf8b295f544d0802cee914e31c384" member_type: VOTER last_known_addr { host: "127.0.206.125" port: 38779 } } peers { permanent_uuid: "5277d3aff87d49a389c54ea34b837cf8" member_type: VOTER last_known_addr { host: "127.0.206.124" port: 40941 } }
I20260619 10:21:25.200925 23617 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5277d3aff87d49a389c54ea34b837cf8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5277d3aff87d49a389c54ea34b837cf8, State: Initialized, Role: FOLLOWER
I20260619 10:21:25.201010 23617 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5277d3aff87d49a389c54ea34b837cf8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3c64103572c4118908cc0f1f778b69a" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 44269 } } peers { permanent_uuid: "cbfcf8b295f544d0802cee914e31c384" member_type: VOTER last_known_addr { host: "127.0.206.125" port: 38779 } } peers { permanent_uuid: "5277d3aff87d49a389c54ea34b837cf8" member_type: VOTER last_known_addr { host: "127.0.206.124" port: 40941 } }
I20260619 10:21:25.201316 23617 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5277d3aff87d49a389c54ea34b837cf8 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 10:21:25.201306 23623 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5277d3aff87d49a389c54ea34b837cf8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3c64103572c4118908cc0f1f778b69a" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 44269 } } peers { permanent_uuid: "cbfcf8b295f544d0802cee914e31c384" member_type: VOTER last_known_addr { host: "127.0.206.125" port: 38779 } } peers { permanent_uuid: "5277d3aff87d49a389c54ea34b837cf8" member_type: VOTER last_known_addr { host: "127.0.206.124" port: 40941 } } }
I20260619 10:21:25.201406 23623 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5277d3aff87d49a389c54ea34b837cf8 [sys.catalog]: This master's current role is: FOLLOWER
W20260619 10:21:25.202972 23634 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 5277d3aff87d49a389c54ea34b837cf8: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 10:21:25.203019 23634 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 10:21:25.203298 23549 master_runner.cc:186] Error getting master registration for 127.0.206.126:44269: OK, instance_id { permanent_uuid: "b3c64103572c4118908cc0f1f778b69a" instance_seqno: 1781864484873941 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260619 10:21:25.205546 23549 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260619 10:21:25.213455 23467 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.206.124" port: 40941 } attempt: 2
Jun 19 10:21:25 dist-test-slave-6pt2 krb5kdc[23386](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864484, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.126@KRBTEST.COM for kudu/127.0.206.124@KRBTEST.COM
I20260619 10:21:25.219808 23620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:21:25.213768 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35280 (local address 127.0.206.124:40941)
0619 10:21:25.213820 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0619 10:21:25.213822 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 10:21:25.213830 (+ 8us) server_negotiation.cc:407] Connection header received
0619 10:21:25.213886 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:21:25.213887 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:21:25.213895 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:21:25.213938 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:21:25.214191 (+ 253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:21:25.214731 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:21:25.215337 (+ 606us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:21:25.215541 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:21:25.215613 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:21:25.217372 (+ 1759us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 10:21:25.217373 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 10:21:25.217374 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 10:21:25.217381 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 10:21:25.218701 (+ 1320us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:21:25.219025 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:21:25.219026 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:21:25.219027 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:21:25.219078 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:21:25.219281 (+ 203us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:21:25.219282 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:21:25.219282 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:21:25.219429 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 10:21:25.219506 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0619 10:21:25.219661 (+ 155us) server_negotiation.cc:299] Negotiation successful
0619 10:21:25.219723 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260619 10:21:25.219812 23469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:21:25.213719 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.124:40941 (local address 127.0.0.1:35280)
0619 10:21:25.213758 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0619 10:21:25.213773 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0619 10:21:25.213826 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 10:21:25.213950 (+ 124us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 10:21:25.213951 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 10:21:25.214085 (+ 134us) client_negotiation.cc:189] Negotiated authn=SASL
0619 10:21:25.214118 (+ 33us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:21:25.214120 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:21:25.214803 (+ 683us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:21:25.214804 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:21:25.215258 (+ 454us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:21:25.215260 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:21:25.215611 (+ 351us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:21:25.215612 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:21:25.215664 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:21:25.216094 (+ 430us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 10:21:25.216099 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 10:21:25.217291 (+ 1192us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 10:21:25.218785 (+ 1494us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 10:21:25.218786 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 10:21:25.218786 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 10:21:25.218954 (+ 168us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 10:21:25.219153 (+ 199us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 10:21:25.219154 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 10:21:25.219154 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 10:21:25.219209 (+ 55us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 10:21:25.219500 (+ 291us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 10:21:25.219501 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 10:21:25.219542 (+ 41us) client_negotiation.cc:769] Sending connection context
0619 10:21:25.219638 (+ 96us) client_negotiation.cc:240] Negotiation successful
0619 10:21:25.219724 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":21}
I20260619 10:21:25.221052 23467 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b3c64103572c4118908cc0f1f778b69a: Bootstrap starting.
I20260619 10:21:25.221570 23467 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b3c64103572c4118908cc0f1f778b69a: Neither blocks nor log segments found. Creating new log.
I20260619 10:21:25.221776 23467 log.cc:824] T 00000000000000000000000000000000 P b3c64103572c4118908cc0f1f778b69a: Log is configured to *not* fsync() on all Append() calls
I20260619 10:21:25.222452 23467 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b3c64103572c4118908cc0f1f778b69a: No bootstrap required, opened a new log
I20260619 10:21:25.223428 23467 raft_consensus.cc:348] T 00000000000000000000000000000000 P b3c64103572c4118908cc0f1f778b69a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3c64103572c4118908cc0f1f778b69a" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 44269 } } peers { permanent_uuid: "cbfcf8b295f544d0802cee914e31c384" member_type: VOTER last_known_addr { host: "127.0.206.125" port: 38779 } } peers { permanent_uuid: "5277d3aff87d49a389c54ea34b837cf8" member_type: VOTER last_known_addr { host: "127.0.206.124" port: 40941 } }
I20260619 10:21:25.223527 23467 raft_consensus.cc:729] T 00000000000000000000000000000000 P b3c64103572c4118908cc0f1f778b69a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b3c64103572c4118908cc0f1f778b69a, State: Initialized, Role: FOLLOWER
I20260619 10:21:25.223580 23467 consensus_queue.cc:260] T 00000000000000000000000000000000 P b3c64103572c4118908cc0f1f778b69a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3c64103572c4118908cc0f1f778b69a" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 44269 } } peers { permanent_uuid: "cbfcf8b295f544d0802cee914e31c384" member_type: VOTER last_known_addr { host: "127.0.206.125" port: 38779 } } peers { permanent_uuid: "5277d3aff87d49a389c54ea34b837cf8" member_type: VOTER last_known_addr { host: "127.0.206.124" port: 40941 } }
I20260619 10:21:25.223793 23637 sys_catalog.cc:455] T 00000000000000000000000000000000 P b3c64103572c4118908cc0f1f778b69a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3c64103572c4118908cc0f1f778b69a" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 44269 } } peers { permanent_uuid: "cbfcf8b295f544d0802cee914e31c384" member_type: VOTER last_known_addr { host: "127.0.206.125" port: 38779 } } peers { permanent_uuid: "5277d3aff87d49a389c54ea34b837cf8" member_type: VOTER last_known_addr { host: "127.0.206.124" port: 40941 } } }
I20260619 10:21:25.223847 23467 sys_catalog.cc:564] T 00000000000000000000000000000000 P b3c64103572c4118908cc0f1f778b69a [sys.catalog]: configured and running, proceeding with master startup.
I20260619 10:21:25.223865 23637 sys_catalog.cc:458] T 00000000000000000000000000000000 P b3c64103572c4118908cc0f1f778b69a [sys.catalog]: This master's current role is: FOLLOWER
I20260619 10:21:25.223946 23441 raft_consensus.cc:482] T 00000000000000000000000000000000 P b3c64103572c4118908cc0f1f778b69a [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260619 10:21:25.224057 23441 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b3c64103572c4118908cc0f1f778b69a [term 0 FOLLOWER]: Advancing to term 1
I20260619 10:21:25.224839 23441 raft_consensus.cc:504] T 00000000000000000000000000000000 P b3c64103572c4118908cc0f1f778b69a [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3c64103572c4118908cc0f1f778b69a" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 44269 } } peers { permanent_uuid: "cbfcf8b295f544d0802cee914e31c384" member_type: VOTER last_known_addr { host: "127.0.206.125" port: 38779 } } peers { permanent_uuid: "5277d3aff87d49a389c54ea34b837cf8" member_type: VOTER last_known_addr { host: "127.0.206.124" port: 40941 } }
I20260619 10:21:25.225700 23591 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b3c64103572c4118908cc0f1f778b69a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "5277d3aff87d49a389c54ea34b837cf8"
W20260619 10:21:25.225773 23648 catalog_manager.cc:1558] T 00000000000000000000000000000000 P b3c64103572c4118908cc0f1f778b69a: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
I20260619 10:21:25.225806 23591 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5277d3aff87d49a389c54ea34b837cf8 [term 0 FOLLOWER]: Advancing to term 1
W20260619 10:21:25.225819 23648 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 10:21:25.226641 23591 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 5277d3aff87d49a389c54ea34b837cf8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b3c64103572c4118908cc0f1f778b69a in term 1.
I20260619 10:21:25.226861 23415 leader_election.cc:302] T 00000000000000000000000000000000 P b3c64103572c4118908cc0f1f778b69a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5277d3aff87d49a389c54ea34b837cf8, b3c64103572c4118908cc0f1f778b69a; no voters:
I20260619 10:21:25.226977 23637 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b3c64103572c4118908cc0f1f778b69a [term 1 FOLLOWER]: Leader election won for term 1
I20260619 10:21:25.227131 23637 raft_consensus.cc:686] T 00000000000000000000000000000000 P b3c64103572c4118908cc0f1f778b69a [term 1 LEADER]: Becoming Leader. State: Replica: b3c64103572c4118908cc0f1f778b69a, State: Running, Role: LEADER
I20260619 10:21:25.227248 23637 consensus_queue.cc:237] T 00000000000000000000000000000000 P b3c64103572c4118908cc0f1f778b69a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3c64103572c4118908cc0f1f778b69a" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 44269 } } peers { permanent_uuid: "cbfcf8b295f544d0802cee914e31c384" member_type: VOTER last_known_addr { host: "127.0.206.125" port: 38779 } } peers { permanent_uuid: "5277d3aff87d49a389c54ea34b837cf8" member_type: VOTER last_known_addr { host: "127.0.206.124" port: 40941 } }
I20260619 10:21:25.227121 23441 leader_election.cc:288] T 00000000000000000000000000000000 P b3c64103572c4118908cc0f1f778b69a [CANDIDATE]: Term 1 election: Requested vote from peers cbfcf8b295f544d0802cee914e31c384 (127.0.206.125:38779), 5277d3aff87d49a389c54ea34b837cf8 (127.0.206.124:40941)
I20260619 10:21:25.227645 23650 sys_catalog.cc:455] T 00000000000000000000000000000000 P b3c64103572c4118908cc0f1f778b69a [sys.catalog]: SysCatalogTable state changed. Reason: New leader b3c64103572c4118908cc0f1f778b69a. Latest consensus state: current_term: 1 leader_uuid: "b3c64103572c4118908cc0f1f778b69a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3c64103572c4118908cc0f1f778b69a" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 44269 } } peers { permanent_uuid: "cbfcf8b295f544d0802cee914e31c384" member_type: VOTER last_known_addr { host: "127.0.206.125" port: 38779 } } peers { permanent_uuid: "5277d3aff87d49a389c54ea34b837cf8" member_type: VOTER last_known_addr { host: "127.0.206.124" port: 40941 } } }
I20260619 10:21:25.227782 23650 sys_catalog.cc:458] T 00000000000000000000000000000000 P b3c64103572c4118908cc0f1f778b69a [sys.catalog]: This master's current role is: LEADER
I20260619 10:21:25.227268 23399 master_runner.cc:186] Error getting master registration for 127.0.206.125:38779: OK, instance_id { permanent_uuid: "cbfcf8b295f544d0802cee914e31c384" instance_seqno: 1781864485022742 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260619 10:21:25.229382 23399 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260619 10:21:25.229940 23652 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 10:21:25.230118 23652 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 10:21:25.231716 23591 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 5277d3aff87d49a389c54ea34b837cf8 [term 1 FOLLOWER]: Refusing update from remote peer b3c64103572c4118908cc0f1f778b69a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 10:21:25.232005 23650 consensus_queue.cc:1035] T 00000000000000000000000000000000 P b3c64103572c4118908cc0f1f778b69a [LEADER]: Connected to new peer: Peer: permanent_uuid: "5277d3aff87d49a389c54ea34b837cf8" member_type: VOTER last_known_addr { host: "127.0.206.124" port: 40941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 10:21:25.233594 23623 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5277d3aff87d49a389c54ea34b837cf8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b3c64103572c4118908cc0f1f778b69a. Latest consensus state: current_term: 1 leader_uuid: "b3c64103572c4118908cc0f1f778b69a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3c64103572c4118908cc0f1f778b69a" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 44269 } } peers { permanent_uuid: "cbfcf8b295f544d0802cee914e31c384" member_type: VOTER last_known_addr { host: "127.0.206.125" port: 38779 } } peers { permanent_uuid: "5277d3aff87d49a389c54ea34b837cf8" member_type: VOTER last_known_addr { host: "127.0.206.124" port: 40941 } } }
I20260619 10:21:25.233690 23623 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5277d3aff87d49a389c54ea34b837cf8 [sys.catalog]: This master's current role is: FOLLOWER
I20260619 10:21:25.234915 23637 sys_catalog.cc:455] T 00000000000000000000000000000000 P b3c64103572c4118908cc0f1f778b69a [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "b3c64103572c4118908cc0f1f778b69a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3c64103572c4118908cc0f1f778b69a" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 44269 } } peers { permanent_uuid: "cbfcf8b295f544d0802cee914e31c384" member_type: VOTER last_known_addr { host: "127.0.206.125" port: 38779 } } peers { permanent_uuid: "5277d3aff87d49a389c54ea34b837cf8" member_type: VOTER last_known_addr { host: "127.0.206.124" port: 40941 } } }
I20260619 10:21:25.235002 23637 sys_catalog.cc:458] T 00000000000000000000000000000000 P b3c64103572c4118908cc0f1f778b69a [sys.catalog]: This master's current role is: LEADER
I20260619 10:21:25.235092 23623 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5277d3aff87d49a389c54ea34b837cf8 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "b3c64103572c4118908cc0f1f778b69a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3c64103572c4118908cc0f1f778b69a" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 44269 } } peers { permanent_uuid: "cbfcf8b295f544d0802cee914e31c384" member_type: VOTER last_known_addr { host: "127.0.206.125" port: 38779 } } peers { permanent_uuid: "5277d3aff87d49a389c54ea34b837cf8" member_type: VOTER last_known_addr { host: "127.0.206.124" port: 40941 } } }
I20260619 10:21:25.235170 23623 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5277d3aff87d49a389c54ea34b837cf8 [sys.catalog]: This master's current role is: FOLLOWER
I20260619 10:21:25.235141 23657 mvcc.cc:205] Tried to move back new op lower bound from 7298516931506499584 to 7298516931491340288. Current Snapshot: MvccSnapshot[applied={T|T < 7298516931506499584}]
I20260619 10:21:25.236049 23661 mvcc.cc:205] Tried to move back new op lower bound from 7298516931506499584 to 7298516931491340288. Current Snapshot: MvccSnapshot[applied={T|T < 7298516931506499584}]
I20260619 10:21:25.240590 23652 catalog_manager.cc:1347] Generated new cluster ID: a5ba3aed4cfe414c81ece2301ca2acf6
I20260619 10:21:25.240638 23652 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 10:21:25.253830 23542 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.206.124" port: 40941 } attempt: 3
I20260619 10:21:25.254650 23652 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 10:21:25.256042 23652 catalog_manager.cc:1504] Loading token signing keys...
I20260619 10:21:25.261368 23546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:21:25.231133 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36214 (local address 127.0.206.125:38779)
0619 10:21:25.231170 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0619 10:21:25.231171 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 10:21:25.233230 (+ 2059us) server_negotiation.cc:407] Connection header received
0619 10:21:25.233344 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:21:25.233345 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:21:25.233355 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:21:25.233383 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:21:25.237246 (+ 3863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:21:25.237933 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:21:25.240026 (+ 2093us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:21:25.240342 (+ 316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:21:25.240379 (+ 37us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:21:25.254718 (+ 14339us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 10:21:25.254720 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 10:21:25.254720 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 10:21:25.254727 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 10:21:25.256747 (+ 2020us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:21:25.258757 (+ 2010us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:21:25.258758 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:21:25.258758 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:21:25.258817 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:21:25.260016 (+ 1199us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:21:25.260017 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:21:25.260017 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:21:25.260146 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 10:21:25.260642 (+ 496us) server_negotiation.cc:1035] Waiting for connection context
0619 10:21:25.261209 (+ 567us) server_negotiation.cc:299] Negotiation successful
0619 10:21:25.261282 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260619 10:21:25.262178 23470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:21:25.230939 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43308 (local address 127.0.206.126:44269)
0619 10:21:25.230974 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0619 10:21:25.230976 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 10:21:25.232284 (+ 1308us) server_negotiation.cc:407] Connection header received
0619 10:21:25.232375 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:21:25.232375 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:21:25.232385 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:21:25.232415 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:21:25.234126 (+ 1711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:21:25.236157 (+ 2031us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:21:25.237217 (+ 1060us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:21:25.237492 (+ 275us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:21:25.237550 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:21:25.255216 (+ 17666us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 10:21:25.255218 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 10:21:25.255218 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 10:21:25.255224 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 10:21:25.257488 (+ 2264us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:21:25.259016 (+ 1528us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:21:25.259017 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:21:25.259018 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:21:25.259064 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:21:25.260801 (+ 1737us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:21:25.260802 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:21:25.260803 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:21:25.260929 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 10:21:25.260997 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0619 10:21:25.261989 (+ 992us) server_negotiation.cc:299] Negotiation successful
0619 10:21:25.262079 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260619 10:21:25.265553 23620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:21:25.232059 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35290 (local address 127.0.206.124:40941)
0619 10:21:25.232104 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0619 10:21:25.232106 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 10:21:25.238759 (+ 6653us) server_negotiation.cc:407] Connection header received
0619 10:21:25.238805 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:21:25.238806 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:21:25.238815 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:21:25.238842 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:21:25.239543 (+ 701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:21:25.240163 (+ 620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:21:25.258107 (+ 17944us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:21:25.258400 (+ 293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:21:25.258441 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:21:25.262783 (+ 4342us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 10:21:25.262784 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 10:21:25.262784 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 10:21:25.262791 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 10:21:25.264484 (+ 1693us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:21:25.264828 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:21:25.264829 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:21:25.264829 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:21:25.264878 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:21:25.265060 (+ 182us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:21:25.265061 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:21:25.265061 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:21:25.265168 (+ 107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 10:21:25.265233 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0619 10:21:25.265400 (+ 167us) server_negotiation.cc:299] Negotiation successful
0619 10:21:25.265468 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
Jun 19 10:21:25 dist-test-slave-6pt2 krb5kdc[23386](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781864484, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.206.125@KRBTEST.COM for kudu/127.0.206.124@KRBTEST.COM
I20260619 10:21:25.275182 23544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:21:25.254085 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.0.206.124:40941 (local address 127.0.0.1:35304)
0619 10:21:25.254170 (+ 85us) negotiation.cc:107] Waiting for socket to connect
0619 10:21:25.254195 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0619 10:21:25.254236 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 10:21:25.263198 (+ 8962us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 10:21:25.263198 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 10:21:25.263340 (+ 142us) client_negotiation.cc:189] Negotiated authn=SASL
0619 10:21:25.263379 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:21:25.263380 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:21:25.266987 (+ 3607us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:21:25.266988 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:21:25.267780 (+ 792us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 10:21:25.267782 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 10:21:25.268260 (+ 478us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 10:21:25.268260 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 10:21:25.268309 (+ 49us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:21:25.268682 (+ 373us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 10:21:25.268686 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 10:21:25.270804 (+ 2118us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 10:21:25.273293 (+ 2489us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 10:21:25.273294 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 10:21:25.273294 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 10:21:25.273472 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 10:21:25.274469 (+ 997us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 10:21:25.274469 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 10:21:25.274470 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 10:21:25.274542 (+ 72us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 10:21:25.274932 (+ 390us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 10:21:25.274933 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 10:21:25.274981 (+ 48us) client_negotiation.cc:769] Sending connection context
0619 10:21:25.275022 (+ 41us) client_negotiation.cc:240] Negotiation successful
0619 10:21:25.275097 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":67}
I20260619 10:21:25.275713 23665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:21:25.254324 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35304 (local address 127.0.206.124:40941)
0619 10:21:25.263057 (+ 8733us) server_negotiation.cc:206] Beginning negotiation
0619 10:21:25.263058 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 10:21:25.263065 (+ 7us) server_negotiation.cc:407] Connection header received
0619 10:21:25.263083 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:21:25.263083 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:21:25.263093 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:21:25.263721 (+ 628us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:21:25.263753 (+ 32us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:21:25.266901 (+ 3148us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:21:25.267881 (+ 980us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:21:25.268185 (+ 304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:21:25.268433 (+ 248us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:21:25.270941 (+ 2508us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 10:21:25.270943 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 10:21:25.270943 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 10:21:25.270948 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0619 10:21:25.273169 (+ 2221us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:21:25.273553 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:21:25.273554 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:21:25.273554 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:21:25.274351 (+ 797us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 10:21:25.274651 (+ 300us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 10:21:25.274651 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 10:21:25.274652 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 10:21:25.274843 (+ 191us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 10:21:25.275382 (+ 539us) server_negotiation.cc:1035] Waiting for connection context
0619 10:21:25.275449 (+ 67us) server_negotiation.cc:299] Negotiation successful
0619 10:21:25.275572 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8715,"thread_start_us":101,"threads_started":1}
I20260619 10:21:25.277261 23542 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cbfcf8b295f544d0802cee914e31c384: Bootstrap starting.
I20260619 10:21:25.277709 23652 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b3c64103572c4118908cc0f1f778b69a: Generated new TSK 0
I20260619 10:21:25.277921 23652 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 10:21:25.278036 23542 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cbfcf8b295f544d0802cee914e31c384: Neither blocks nor log segments found. Creating new log.
I20260619 10:21:25.278334 23542 log.cc:824] T 00000000000000000000000000000000 P cbfcf8b295f544d0802cee914e31c384: Log is configured to *not* fsync() on all Append() calls
I20260619 10:21:25.279151 23542 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cbfcf8b295f544d0802cee914e31c384: No bootstrap required, opened a new log
I20260619 10:21:25.281584 23542 raft_consensus.cc:348] T 00000000000000000000000000000000 P cbfcf8b295f544d0802cee914e31c384 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3c64103572c4118908cc0f1f778b69a" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 44269 } } peers { permanent_uuid: "cbfcf8b295f544d0802cee914e31c384" member_type: VOTER last_known_addr { host: "127.0.206.125" port: 38779 } } peers { permanent_uuid: "5277d3aff87d49a389c54ea34b837cf8" member_type: VOTER last_known_addr { host: "127.0.206.124" port: 40941 } }
I20260619 10:21:25.281790 23542 raft_consensus.cc:729] T 00000000000000000000000000000000 P cbfcf8b295f544d0802cee914e31c384 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cbfcf8b295f544d0802cee914e31c384, State: Initialized, Role: FOLLOWER
I20260619 10:21:25.281926 23542 consensus_queue.cc:260] T 00000000000000000000000000000000 P cbfcf8b295f544d0802cee914e31c384 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3c64103572c4118908cc0f1f778b69a" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 44269 } } peers { permanent_uuid: "cbfcf8b295f544d0802cee914e31c384" member_type: VOTER last_known_addr { host: "127.0.206.125" port: 38779 } } peers { permanent_uuid: "5277d3aff87d49a389c54ea34b837cf8" member_type: VOTER last_known_addr { host: "127.0.206.124" port: 40941 } }
I20260619 10:21:25.282290 23542 sys_catalog.cc:564] T 00000000000000000000000000000000 P cbfcf8b295f544d0802cee914e31c384 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 10:21:25.283326 23516 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b3c64103572c4118908cc0f1f778b69a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "cbfcf8b295f544d0802cee914e31c384"
I20260619 10:21:25.283459 23516 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cbfcf8b295f544d0802cee914e31c384 [term 0 FOLLOWER]: Advancing to term 1
I20260619 10:21:25.284332 23516 raft_consensus.cc:2454] T 00000000000000000000000000000000 P cbfcf8b295f544d0802cee914e31c384 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b3c64103572c4118908cc0f1f778b69a in term 1.
I20260619 10:21:25.284569 23515 raft_consensus.cc:1261] T 00000000000000000000000000000000 P cbfcf8b295f544d0802cee914e31c384 [term 1 FOLLOWER]: Refusing update from remote peer b3c64103572c4118908cc0f1f778b69a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 10:21:25.284833 23650 consensus_queue.cc:1035] T 00000000000000000000000000000000 P b3c64103572c4118908cc0f1f778b69a [LEADER]: Connected to new peer: Peer: permanent_uuid: "cbfcf8b295f544d0802cee914e31c384" member_type: VOTER last_known_addr { host: "127.0.206.125" port: 38779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 10:21:25.286950 23667 sys_catalog.cc:455] T 00000000000000000000000000000000 P cbfcf8b295f544d0802cee914e31c384 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b3c64103572c4118908cc0f1f778b69a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3c64103572c4118908cc0f1f778b69a" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 44269 } } peers { permanent_uuid: "cbfcf8b295f544d0802cee914e31c384" member_type: VOTER last_known_addr { host: "127.0.206.125" port: 38779 } } peers { permanent_uuid: "5277d3aff87d49a389c54ea34b837cf8" member_type: VOTER last_known_addr { host: "127.0.206.124" port: 40941 } } }
I20260619 10:21:25.287218 23667 sys_catalog.cc:458] T 00000000000000000000000000000000 P cbfcf8b295f544d0802cee914e31c384 [sys.catalog]: This master's current role is: FOLLOWER
I20260619 10:21:25.287317 23667 sys_catalog.cc:455] T 00000000000000000000000000000000 P cbfcf8b295f544d0802cee914e31c384 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b3c64103572c4118908cc0f1f778b69a. Latest consensus state: current_term: 1 leader_uuid: "b3c64103572c4118908cc0f1f778b69a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3c64103572c4118908cc0f1f778b69a" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 44269 } } peers { permanent_uuid: "cbfcf8b295f544d0802cee914e31c384" member_type: VOTER last_known_addr { host: "127.0.206.125" port: 38779 } } peers { permanent_uuid: "5277d3aff87d49a389c54ea34b837cf8" member_type: VOTER last_known_addr { host: "127.0.206.124" port: 40941 } } }
I20260619 10:21:25.287428 23667 sys_catalog.cc:458] T 00000000000000000000000000000000 P cbfcf8b295f544d0802cee914e31c384 [sys.catalog]: This master's current role is: FOLLOWER
I20260619 10:21:25.287494 23667 sys_catalog.cc:455] T 00000000000000000000000000000000 P cbfcf8b295f544d0802cee914e31c384 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "b3c64103572c4118908cc0f1f778b69a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3c64103572c4118908cc0f1f778b69a" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 44269 } } peers { permanent_uuid: "cbfcf8b295f544d0802cee914e31c384" member_type: VOTER last_known_addr { host: "127.0.206.125" port: 38779 } } peers { permanent_uuid: "5277d3aff87d49a389c54ea34b837cf8" member_type: VOTER last_known_addr { host: "127.0.206.124" port: 40941 } } }
I20260619 10:21:25.287583 23667 sys_catalog.cc:458] T 00000000000000000000000000000000 P cbfcf8b295f544d0802cee914e31c384 [sys.catalog]: This master's current role is: FOLLOWER
I20260619 10:21:25.289139 23680 mvcc.cc:205] Tried to move back new op lower bound from 7298516931692257280 to 7298516931491340288. Current Snapshot: MvccSnapshot[applied={T|T < 7298516931506499584}]
W20260619 10:21:25.289976 23683 catalog_manager.cc:1558] T 00000000000000000000000000000000 P cbfcf8b295f544d0802cee914e31c384: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 10:21:25.290027 23683 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260619 10:21:25.306005 23470 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 10:21:25.304137 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43312 (local address 127.0.206.126:44269)
0619 10:21:25.304175 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0619 10:21:25.304177 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 10:21:25.305099 (+ 922us) server_negotiation.cc:407] Connection header received
0619 10:21:25.305504 (+ 405us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:21:25.305505 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:21:25.305514 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:21:25.305541 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:21:25.305910 (+ 369us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43312: BlockingRecv error: recv got EOF from 127.0.0.1:43312 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
W20260619 10:21:25.306615 23546 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 10:21:25.304320 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36216 (local address 127.0.206.125:38779)
0619 10:21:25.304352 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0619 10:21:25.304353 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 10:21:25.305018 (+ 665us) server_negotiation.cc:407] Connection header received
0619 10:21:25.305195 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:21:25.305196 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:21:25.305205 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:21:25.305249 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:21:25.306549 (+ 1300us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36216: BlockingRecv error: recv got EOF from 127.0.0.1:36216 (error 108)
Metrics: {"server-negotiator.queue_time_us":16}
W20260619 10:21:25.307452 23665 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 10:21:25.304777 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35312 (local address 127.0.206.124:40941)
0619 10:21:25.304812 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0619 10:21:25.304814 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 10:21:25.306918 (+ 2104us) server_negotiation.cc:407] Connection header received
0619 10:21:25.307063 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:21:25.307064 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:21:25.307073 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:21:25.307102 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:21:25.307384 (+ 282us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35312: BlockingRecv error: recv got EOF from 127.0.0.1:35312 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
W20260619 10:21:25.309656 23546 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 10:21:25.308939 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36230 (local address 127.0.206.125:38779)
0619 10:21:25.308970 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0619 10:21:25.308971 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 10:21:25.309175 (+ 204us) server_negotiation.cc:407] Connection header received
0619 10:21:25.309270 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:21:25.309271 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:21:25.309285 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:21:25.309313 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:21:25.309616 (+ 303us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36230: BlockingRecv error: recv got EOF from 127.0.0.1:36230 (error 108)
Metrics: {"server-negotiator.queue_time_us":15}
W20260619 10:21:25.312549 23546 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 10:21:25.311851 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36232 (local address 127.0.206.125:38779)
0619 10:21:25.311880 (+ 29us) server_negotiation.cc:206] Beginning negotiation
0619 10:21:25.311882 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 10:21:25.312087 (+ 205us) server_negotiation.cc:407] Connection header received
0619 10:21:25.312175 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:21:25.312176 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:21:25.312185 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:21:25.312212 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:21:25.312511 (+ 299us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36232: BlockingRecv error: recv got EOF from 127.0.0.1:36232 (error 108)
Metrics: {"server-negotiator.queue_time_us":15}
W20260619 10:21:25.316336 23546 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 10:21:25.315637 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36248 (local address 127.0.206.125:38779)
0619 10:21:25.315666 (+ 29us) server_negotiation.cc:206] Beginning negotiation
0619 10:21:25.315667 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 10:21:25.315858 (+ 191us) server_negotiation.cc:407] Connection header received
0619 10:21:25.315961 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:21:25.315962 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:21:25.315989 (+ 27us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:21:25.316017 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:21:25.316297 (+ 280us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36248: BlockingRecv error: recv got EOF from 127.0.0.1:36248 (error 108)
Metrics: {"server-negotiator.queue_time_us":15}
W20260619 10:21:25.322114 23546 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 10:21:25.321402 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36262 (local address 127.0.206.125:38779)
0619 10:21:25.321435 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0619 10:21:25.321437 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 10:21:25.321639 (+ 202us) server_negotiation.cc:407] Connection header received
0619 10:21:25.321723 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:21:25.321723 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:21:25.321732 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:21:25.321758 (+ 26us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:21:25.322077 (+ 319us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36262: BlockingRecv error: recv got EOF from 127.0.0.1:36262 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
W20260619 10:21:25.331890 23546 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 10:21:25.331224 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36276 (local address 127.0.206.125:38779)
0619 10:21:25.331253 (+ 29us) server_negotiation.cc:206] Beginning negotiation
0619 10:21:25.331254 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 10:21:25.331456 (+ 202us) server_negotiation.cc:407] Connection header received
0619 10:21:25.331541 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:21:25.331542 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:21:25.331551 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:21:25.331577 (+ 26us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:21:25.331842 (+ 265us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36276: BlockingRecv error: recv got EOF from 127.0.0.1:36276 (error 108)
Metrics: {"server-negotiator.queue_time_us":14}
W20260619 10:21:25.349864 23546 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 10:21:25.349077 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36282 (local address 127.0.206.125:38779)
0619 10:21:25.349131 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0619 10:21:25.349133 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 10:21:25.349388 (+ 255us) server_negotiation.cc:407] Connection header received
0619 10:21:25.349477 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:21:25.349479 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:21:25.349489 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:21:25.349516 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:21:25.349812 (+ 296us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36282: BlockingRecv error: recv got EOF from 127.0.0.1:36282 (error 108)
Metrics: {"server-negotiator.queue_time_us":36}
W20260619 10:21:25.384466 23546 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 10:21:25.383464 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36286 (local address 127.0.206.125:38779)
0619 10:21:25.383506 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0619 10:21:25.383509 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 10:21:25.383787 (+ 278us) server_negotiation.cc:407] Connection header received
0619 10:21:25.383896 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:21:25.383897 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:21:25.383922 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:21:25.383954 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:21:25.384401 (+ 447us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36286: BlockingRecv error: recv got EOF from 127.0.0.1:36286 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
W20260619 10:21:25.450687 23546 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 10:21:25.450017 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36290 (local address 127.0.206.125:38779)
0619 10:21:25.450129 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0619 10:21:25.450131 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 10:21:25.450142 (+ 11us) server_negotiation.cc:407] Connection header received
0619 10:21:25.450216 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:21:25.450217 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:21:25.450227 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:21:25.450276 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:21:25.450588 (+ 312us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36290: BlockingRecv error: recv got EOF from 127.0.0.1:36290 (error 108)
Metrics: {"server-negotiator.queue_time_us":89}
W20260619 10:21:25.580448 23546 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 10:21:25.579743 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36292 (local address 127.0.206.125:38779)
0619 10:21:25.579813 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0619 10:21:25.579815 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 10:21:25.579930 (+ 115us) server_negotiation.cc:407] Connection header received
0619 10:21:25.579997 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:21:25.579998 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:21:25.580007 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:21:25.580047 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:21:25.580365 (+ 318us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36292: BlockingRecv error: recv got EOF from 127.0.0.1:36292 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
I20260619 10:21:25.802548 23741 sys_catalog.cc:455] T 00000000000000000000000000000000 P b3c64103572c4118908cc0f1f778b69a [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "b3c64103572c4118908cc0f1f778b69a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3c64103572c4118908cc0f1f778b69a" member_type: VOTER last_known_addr { host: "127.0.206.126" port: 44269 } } peers { permanent_uuid: "cbfcf8b295f544d0802cee914e31c384" member_type: VOTER last_known_addr { host: "127.0.206.125" port: 38779 } } peers { permanent_uuid: "5277d3aff87d49a389c54ea34b837cf8" member_type: VOTER last_known_addr { host: "127.0.206.124" port: 40941 } } }
I20260619 10:21:25.802717 23741 sys_catalog.cc:458] T 00000000000000000000000000000000 P b3c64103572c4118908cc0f1f778b69a [sys.catalog]: This master's current role is: LEADER
W20260619 10:21:25.838157 23546 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 10:21:25.837583 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36294 (local address 127.0.206.125:38779)
0619 10:21:25.837629 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0619 10:21:25.837631 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 10:21:25.837671 (+ 40us) server_negotiation.cc:407] Connection header received
0619 10:21:25.837731 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:21:25.837731 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:21:25.837740 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:21:25.837789 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0619 10:21:25.838100 (+ 311us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36294: BlockingRecv error: recv got EOF from 127.0.0.1:36294 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
I20260619 10:21:26.203409 23634 catalog_manager.cc:1259] Loaded cluster ID: a5ba3aed4cfe414c81ece2301ca2acf6
I20260619 10:21:26.203470 23634 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 5277d3aff87d49a389c54ea34b837cf8: loading cluster ID for follower catalog manager: success
I20260619 10:21:26.204267 23634 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 5277d3aff87d49a389c54ea34b837cf8: acquiring CA information for follower catalog manager: success
I20260619 10:21:26.204584 23634 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 5277d3aff87d49a389c54ea34b837cf8: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260619 10:21:26.290408 23683 catalog_manager.cc:1259] Loaded cluster ID: a5ba3aed4cfe414c81ece2301ca2acf6
I20260619 10:21:26.290488 23683 catalog_manager.cc:1552] T 00000000000000000000000000000000 P cbfcf8b295f544d0802cee914e31c384: loading cluster ID for follower catalog manager: success
I20260619 10:21:26.291272 23683 catalog_manager.cc:1574] T 00000000000000000000000000000000 P cbfcf8b295f544d0802cee914e31c384: acquiring CA information for follower catalog manager: success
I20260619 10:21:26.291517 23683 catalog_manager.cc:1602] T 00000000000000000000000000000000 P cbfcf8b295f544d0802cee914e31c384: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260619 10:21:26.353646 23754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:21:26.351329 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36308 (local address 127.0.206.125:38779)
0619 10:21:26.351466 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0619 10:21:26.351468 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 10:21:26.351504 (+ 36us) server_negotiation.cc:407] Connection header received
0619 10:21:26.351537 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:21:26.351537 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:21:26.351582 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:21:26.351643 (+ 61us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 10:21:26.351768 (+ 125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:21:26.352320 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:21:26.353027 (+ 707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:21:26.353199 (+ 172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:21:26.353245 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:21:26.353389 (+ 144us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 10:21:26.353485 (+ 96us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 10:21:26.353540 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0619 10:21:26.353575 (+ 35us) server_negotiation.cc:299] Negotiation successful
0619 10:21:26.353589 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":112,"thread_start_us":92,"threads_started":1}
I20260619 10:21:26.357517 23760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:21:26.355469 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35314 (local address 127.0.206.124:40941)
0619 10:21:26.355574 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0619 10:21:26.355576 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 10:21:26.355583 (+ 7us) server_negotiation.cc:407] Connection header received
0619 10:21:26.355625 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:21:26.355626 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:21:26.355655 (+ 29us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:21:26.355716 (+ 61us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 10:21:26.355789 (+ 73us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:21:26.356258 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:21:26.356892 (+ 634us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:21:26.357074 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:21:26.357122 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:21:26.357233 (+ 111us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 10:21:26.357324 (+ 91us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 10:21:26.357374 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0619 10:21:26.357416 (+ 42us) server_negotiation.cc:299] Negotiation successful
0619 10:21:26.357431 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88,"thread_start_us":66,"threads_started":1}
I20260619 10:21:26.361822 23754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:21:26.359127 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36316 (local address 127.0.206.125:38779)
0619 10:21:26.359162 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0619 10:21:26.359164 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 10:21:26.359400 (+ 236us) server_negotiation.cc:407] Connection header received
0619 10:21:26.359426 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:21:26.359427 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:21:26.359436 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:21:26.359489 (+ 53us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 10:21:26.359593 (+ 104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:21:26.360138 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:21:26.361086 (+ 948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:21:26.361384 (+ 298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:21:26.361444 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:21:26.361608 (+ 164us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 10:21:26.361638 (+ 30us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 10:21:26.361683 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0619 10:21:26.361735 (+ 52us) server_negotiation.cc:299] Negotiation successful
0619 10:21:26.361752 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260619 10:21:26.362030 23760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:21:26.359325 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35316 (local address 127.0.206.124:40941)
0619 10:21:26.359380 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0619 10:21:26.359382 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 10:21:26.359424 (+ 42us) server_negotiation.cc:407] Connection header received
0619 10:21:26.359509 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:21:26.359509 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:21:26.359518 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:21:26.359567 (+ 49us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 10:21:26.359718 (+ 151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:21:26.360382 (+ 664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:21:26.361281 (+ 899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:21:26.361569 (+ 288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:21:26.361624 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:21:26.361782 (+ 158us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 10:21:26.361811 (+ 29us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 10:21:26.361840 (+ 29us) server_negotiation.cc:1035] Waiting for connection context
0619 10:21:26.361933 (+ 93us) server_negotiation.cc:299] Negotiation successful
0619 10:21:26.361960 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260619 10:21:26.362366 23767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 10:21:26.359201 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43316 (local address 127.0.206.126:44269)
0619 10:21:26.359534 (+ 333us) server_negotiation.cc:206] Beginning negotiation
0619 10:21:26.359536 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 10:21:26.359543 (+ 7us) server_negotiation.cc:407] Connection header received
0619 10:21:26.359590 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 10:21:26.359591 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 10:21:26.359636 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 10:21:26.359705 (+ 69us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 10:21:26.359881 (+ 176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:21:26.360564 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:21:26.361456 (+ 892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 10:21:26.361758 (+ 302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 10:21:26.361811 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 10:21:26.362031 (+ 220us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 10:21:26.362128 (+ 97us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 10:21:26.362187 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0619 10:21:26.362252 (+ 65us) server_negotiation.cc:299] Negotiation successful
0619 10:21:26.362277 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":314,"thread_start_us":52,"threads_started":1}
I20260619 10:21:26.364065 825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTU9OP6/build/release/bin/kudu with pid 23399
I20260619 10:21:26.369477 825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTU9OP6/build/release/bin/kudu with pid 23474
I20260619 10:21:26.374092 825 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskTU9OP6/build/release/bin/kudu with pid 23549
2026-06-19T10:21:26Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (2741 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (5560 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (261803 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST